The answer is the census work is done by a large number of part time workers. They do not put in in 8 hour days for 5 days a week. Some do, but the overwhelming majority don't. It's more like 4 hours today, 0 hours tomorrow, and maybe 0 hours the day after that ~ and sometimes there are breaks for weeks.
It depends on how many part timers they've got, and where the work takes place.
Even part time, how many hours does it take 1.2 million to count only a small portion of 310 million - since (as USS Alaska pointed out) the Census only sends out "workers" to families that did not answer and mail back their Census forms?
If that small portion is 10-15%, or 31-46 million, that comes to 26-39 people to count for each "worker." Subtracting even 50% of the Census staff for "management" and "support" staff and that's still 52-78 people to count for each actual Census counter!
AMAZING!