Even my first native Polish teacher never explained it!!
It's lexically conditioned, sort of like different plural endings in German. Learn the most forms and then guess when you need to.
A few nouns (most famously krawat) seem to be able to take either....
Also -u tends to block facultative animacy (treating an inanimate noun as if it were animate).