I would deduce from eg. the wikipedia entry
Do 1989 PZPR miała charakter partii państwowej, która sprawowała władzę absolutną
that the correct form - and therefore most commonly seen in written text - is the feminine singular, agreeing as you say with Partia. 'PRL' behaves much the same way inasmuch as you hear PRL-u, etc, but formal written genres use a fem. sing.
I think it has something to do with the fact that grammar in speech is perceived phonologically, ie. /peerel/ just sounds
masculine owing to the final /l/ (you've no doubt seen the adjective peerelowski
which seems to support the idea of this 'phonological' level of grammatical perception).
What I've never heard - and native speakers please correct me if I'm out here - is a plural, whereas English can equivocate between seeing a party as a collection of people and therefore is declined plural, versus the more 'correct' singular. One of my many, persistent errors in spoken Polish, this one!