How could they be called "colored people".
I don't know the origins of the phrase, but it was used in the NAACP (National Association for the Advancement of Colored People) the main organization pushing for civil rights.....
You might hear it into the early 70s though IIRC Nеgro overtook it sometime in the 1950s...
Black and the Afro-American happened around the same time (late 1960s early 70s IIRC) not really sure which was first.....