Ivonashaver....I can understand the frustration, Ive experienced the same thing on a house purchase because your mother married they look into your fathers line.
If you know where and when your father was born, get a copy of his birth cert. this will have your granddads full name, and your grandmothers maiden name. From there you could check army records to see if perhaps he served in the forces, and these record hold a lot of information.
As sugested look into your Grandads things you could find a Aliens Registration Book when he arrived in Sweeden, or perhaps when he arrived in England, these have date and place of birth registered in them.
The big problem is Polish authorities demand original copies (so get two or three when ordering).These will have to be translated by a sworn translator in Poland, they did not accept translations from a English Buero.
Its a long and difficult road and you may ask yourself.......Why do I want a Polish passport?