Thank you for your quick response. What does this '5 year EU residence permit' exactly mean? I tried to Google it but no luck so far. I have a Dutch passport, and i'm born there as well.
Essentially, you need to have a permit from the provincial government that allows you to stay in Poland. While you don't need this if you want to stay here as an EU citizen (legally you do, but they don't care less if you don't have it) - it's seen as the 'proof' that you're residing here for more than 185 days.
britishinpoland.com/blog/2008/03/5-year-registration-karta-pobytu/ explains the process a bit more. Basically, without this, you cannot sit the driving test - and while the actual requirements vary a bit from province to province, the burden is still on you to prove that you intend to stay in Poland.
Effectively, it's not worth considering Poland - the time and effort just isn't worth it if you don't actually live here. There's also the issue that you'll need a sworn translator for the exam - and getting a Dutch-Polish translator could be incredibly difficult.
The law was changed to counter the fact that many Germans were choosing to come to Poland and sit their driving test here - so blame them ;)