Wroclaw gave some excellent advice to follow!
Gregorz might be just a different version of Gregory. When dealing with the past, I like to keep in mind that what someone who has been there knows often trumps conventional wisdom.
I'm not talking from convential wisdom, I'm telling you that Grzegorz is the official name in Polish for Gregory, no other variation is that name. Hence why your grandfathers name seems foreign. Like Wroclaw stated, he had a Jewish name in a Catholic country, it stood out, that's why he likely compared it to Grzegorz. This also leads to another point that I didnt want to get started on, to get flamed. A man with a jewish name, leaving Poland during the 1930s when most Jews fled, living in eastern PL where most Jews lived. Lots of items in common for you to research to be sure you know your family for real.
But remember that some members of the family have secrets. Among these can be something embarrassing like a serious crime, suicide, or a child born out of wedlock.
One of these secrets may be your grandfathers family. It was quite common to change your name for fear of being labeled Jewish. While you are pressing that he isn't. He may have been, and decided that secret will stay with him, or he is the first in his family to announce being atheist... doesn't mean the family line was. Keep an open mind when doing your research. If you press that on items you are unsure about, you wont get far in finding the truth.
When Poles came to the US, few came during the 1930 period, that's why this is setting off a flag. Also, Poles upon arrival would shorten their names, from Kowalski they could have changed to Kowal, or Koval, or possibly changed to the american word for their last name's translation. Transitions from Gersom to Gregory more likely, than to Harry. Harry is totally different, kind of what Jews did to hide the fact they were of Jewish descent.
How can you have a Jewish name? Judaism is a religion. He was from Poland, not Israel.
Are you serious? All jews do not reside in Israel, there was no Israel at the time your grandfather left Poland. Jews were dispersed all around Europe. They typically chose old names from the first testament. Gersom is not a polish name, very rare actually. A catholic polish mother in the early 1900s would not just pick a name like this, hence getting back to the topic to have an open mind in your research. Take the advice Wroclaw provided on checking out various sites & records. Do you know the names of Gersom's parents?