Sure. It was slave work. Like rowing a galley, or guarding the sultan's seraglio. Honest people didn't do it.
Whites weren't interested in competing with blacks for jobs as maids or cooks or field hands and we all know it.
That "and we all know it" crack gives you away. You're letting your spleen show again.
Whites weren't interested in competing with indentured servants as henchboys and scullery-maids, either. So what?
You keep wanting to do it, so just go ahead and do it: "It was all about race because you Southerners are all racists and everybody who's better than you are knows it, nyah, nyah, nyah!" Go ahead, put it up there. You know you want to.
Well, if that's your position then who am I to argue with it?