My impression is that using państwo together with a verb in the second person (plural) is an acceptable semi-formal form of address (I often heard it in PRL-era movies), whereas using pan/pani with the second person (singular) is really wrong and evokes poor education
In contemporary language, using second p. in sing is not polite, however it used to be ok in the past (see. e.g. texts like Bible or historical novels or movies e.g. Sienkiewicz's). In today's language in singular we use it only the way Koala described (to express annoyance or disprespect), but it can be felt as strongly impolite.
On the other hand, in the past using plural with sec. pers. (Wy) was a polite way to refer to other person or persons in both sing an plural (instead of Pan/Pani/Państwo). This is still visible in e.g. commies era movies, and some older people still use it when talking to e.g. priests or elders (I often heard my grandma using it when talking to her mum).
I suspect that this is why using second person in plural is felt as acceptable. Probably it just reminds that old way of referrig the interlocutor (Wy).