chodźmy vs idźmy?

Chodzmy = let's go
Idziemy = we are going/leaving

These are IDIOMATIC usages. Use them the way native speakers do and you'll be fine. They don't follow preconceived notions of logic.