should I move there
No.
e I do like Poland very much
Irrelevant
I don't speak the language
Deal breaker (major deal breaker).
a lot of people speak English anyway
Irrelevant.
is it really that hard to live in a different country
Yes. worth it, if you can adapt, but nothing you've written so far indicates that you can.
I know work will be a big problem for me to find
Yes.
any advice would be appreciated.....
Stay where you are or learn Polish (seriously, you need to be fluent to be happy in Poland long term).