The reason for this is that it will depend on where in the US the person is from as I have difficulties understanding some Americans, however that doesn't have anything to do with speaking too fast as on the whole I feel they speak slow. but more to do with a thick accent.
Yes, for example, I have difficulties with understanding the Cajun accent.
As a Pole in Poland, I find it pretty difficult to understand Americans speaking English. This is because I have been trained to use the "true" British English [chiefly according to the BBC "pronounciation pattern"] from the age of 13.
I've been trained to use British English too since high school but that was the British queen's English (I was even told by one teacher at one point that I talk like her lol). But that's not how normal people talk in the UK, apparently. So it's so much easier for me to understand American English.
And Monitor is right - it's thanks to the American films on Polish TV.
I quite often find it difficult to understand foreign, non-English speakers speaking English if they speak with their own, (too) national accent.
Yes, me too. But I guess it depends how often you hear it. If you hear it often enough you learn to understand it, I think.
Specifically, do they think Americans speak too fast?
No, Americans usually speak more slowly and clearly, than the British, for example.
Not always, though - I remember watching "Gilmore Girls" in original and Lorelai and Rory could speak really fast when they were talking with each other and throwing some cultural references in and sometimes I had problems with understanding them.
So, who knows, maybe you're one of such fast talkers :)