Sorry to hear about the heartache you're going through.
Are you Polish as well? It's hard to tell if he's truly in love but we, guys aren't that much different from you ladies. Sometimes things happen slowly and sometimes they happen with a bang. I'd imagine that he's infatuated right now; the ability to talk to her in Polish while in Poland probably added to the overall feeling of being connected to her.
If I were you I'd let him know that you love him and believe in the two of you but then I'd leave him alone. The fact he was so upfront and honest with you tells me two things - he's a good person and he respects you.
You talking to him about her does NOT help, you're obviously very biased on the issue and he knows that. It's very possible that in a week or two he'll snap out of it but who knows?
It's sad you have to be going through this after sharing two years with him but I guess that's the curse of love. When it's good, it's this breathtaking and toe-curling, wonderful experience amplified by the candle-light sparkles of happiness.
When it's bad however it's the death of love as we know it and it feels like nothing and no one will ever be able to bring this happiness back to our hearts again...
At this point I think you need to back off and let him figure things out and while you do it keep living your life, keep breathing and stay with your friends as much as you can and "simply" hope for the best. That's all you can do. Sincerely wish you the very best.
---
PS. I agree with Softsong's message above with one exception.
Ultimatums to us men are equivalent to declarations of war - they do NOT work - EVER!
In my view it'd be a guaranteed break-up. To tell him "I will be seeing other men" is equivalent to saying "good-bye." Instead I'd say "I love you and hope the two of us have a future together but in the mean time I'll have to plan for my own future as well, one that maybe won't include you, it's up to you now." More of a gentle approach.
The way I understand it he hasn't been unfaithful yet he chose to be upfront with you. No reason to confront him with the "I'll be seeing other men" ultimatum - is so, bye bye it is. Just my testosterone-influenced view on it...
...PS: Everyone knew he had a girlfriend, and the two did nothing, not even kiss, cause she reminded him that he has a girlfriend ... and she said I miss you too, the little slut...
Forgot to mention. I know you don't want to hear this but she is not a slut. Ask yourself if you were in her shoes, would you too say - "sorry, I can't, you already have a girlfriend"?
Many women (and men) wouldn't if they met someone they're attracted to. Sad but true.