We spoke to one family we saw outside their newly built house and they said they bought the plot of land as Rolna and paid to convert it to Budowlana and this process took a few weeks.
That's not likely at all. You can't just 'pay' to convert land within weeks, otherwise every single piece of farmland would be converted overnight and the value increased considerably. You can read the real situation here - mojbiznesnieruchomosci.pl/koszty-przeksztalcenia-dzialki-rolnej-w-budowlana - as you can see, it can be cheap, but only if the conditions are met. Otherwise the zoning plan needs to be changed, and that's a bureaucratic nightmare.
By the way, the new restrictions on farmland means that you can't purchase it anyway.
PS: Are you aware of what you're getting yourself into by trying to build from scratch? There are plenty of excellent offers available from developers right now for houses, and unless you have a background in construction/architecture, I wouldn't try and build it yourself. There's a lack of quality construction workers in Poland, and the good ones are booked months in advance. The other thing - banks are not very happy to lend money for house construction.