They built a garage 3 metres from our fence which we didn't object to, I know I could of.
If their garage`s door or windows face your fence, it should be 4 metres. If they don`t, 3 metres is enough. There are still exceptions to it (length of the garage) which allow to build such structures within 1.5 metre or less from the border.
One can read about it here
They have told me I cannot use this as it could stain their paint on the wall of the garage.
Well, there is no such direct law but if the damage appears, they might demand compensation, indeed.
I do pity you coz you gained lousy nitpicking neighbours.