Whenever I greet him he puts on this huge smile, looks me in the eyes and quickly looks downwards.
He sounds a little shy or is a bit intimidated by your looks. Looks like you're just gonna have to go over and say Hi!
I remember that during my stay in Poland people gave me stares, but at that time, the amount of tourists was zero and they could tell youre abroad from miles away.
Funny enough my one friend just recently returned from Poland to the States. He says girls immediately noticed him on the street and could tell he was from another country, was well dressed, took take of himself. He said the average Polish man (in Poland) in his late 20s or 30s looks like a slob. Wears old out of style clothing, has a big beer belly, doesn't use deodorant and sits in a bar drinking that one beer for like 2 hours.