If there is a vowel before the cosonant(s) in the infinitive suffix, then you use "-em". Otherwise "-ę".
For the first person singular which is what the OP is discussing, I think this would be right a lot of the time.
But for the verb Czytać which you mentioned, the ending would be 'am', Ja czytam, so not '-em' or '-ę', or have I misunderstood you somewhere?
I think it would be worthwhile for the OP to try and understand the classes though, as endings change less reliably with 2nd, 3rd person singular etc.