Yeah, but it's also because they aren't Gypsy/Rom.
Sure, but the distinction isn't usually made by most of the settled population.
The categories in Ireland tend to be:
Traveller (Irish, but of uncertain true origin, make up most of the "non-settled" community here)
Gypsy (derogatory unless applied to a real gypsy, often known as an "English gypsy", who may or may not have traveller links - they sometimes
intermarry - and who may have historical continental roma links)
Roma (distinctive, swarthy, dark clothing, hats etc)
I'm not saying it's right but that is largely the way it is viewed.
I know a few travellers and English gypsies - culturally and lifestyle wise they appear to be very similar to me.