Can anyone do it?
Quite likely, but several of those forms are so rarely used, I believe that one may be forgiven for the occasional mistake!
My main problem is understanding the other person.
This. Particularly the bit in bold.
For example, I can read Czech as fast as Polish or English, and can translate from Czech to Polish or English (particularly the former) quickly. But I have to listen very carefully to the spoken language to fully understand what I'm being told, even though I understand probably 80% of the words. It's why phrasebooks alone are never enough.
Slightly off-topic (though its been mentioned): I find it interesting that someone else has mentioned that Chinese isn't as difficult to speak as many think. I used to go out with someone who was English but used to work in Hong Kong and was fluent in Chinese, and she said the same.
I'm planning on starting to learn the language in a few weeks time, as I'm planning on going to HK myself at the end of the year, and don't trust the "most people speak some English" descriptions I keep reading.
Unfortunately, I can only remember about three characters so far (mind you, I haven't really tried learning yet, lol) so seeing if I can actually read any of it after 6 months will be interesting to say the least! I may report back here though, hehe ;)