I'm not saying poland is 100% safe but compared to other countries including western Europe overall violent crime and victimization rates are lower.
I seem to remember that one issue is that reporting rates are much lower in Poland, so whereas someone in Sweden would be quick to report an assault, in Poland, the same victim would face all sorts of issues with reporting it, especially if it's at the hands of the spouse.
Of course, we shouldn't ignore the issues elsewhere - I'm very much in favour of a "zero tolerance" approach to non-EU citizens and crime. Having said that, the majority of the trouble caused by darker-skinned people in PL seems to be Italians and Spanish guys molesting Polish women rather than Arabs or otherwise.