I was under the impression that we were talking about eliminating the jobs of a lot more than "a few people". I think the objective of automation technology should not be the elimination of those jobs, but shortening the time it takes new technology and innovations to work their way down the stack.
Each of the factories is a few people, they add up to a lot when they start hitting every single mass produced industry. Add in self driving trucks so you don’t need people to deliver those goods, and 3D printers so you aren’t even physically delivering most of your goods and a lot of people are out of work.
The goal of automation is to produce more stuff faster for less money. It’s not trying to eliminate jobs, but it inevitably will. That’s a significant part of “less money”, employees are expensive, even if you don’t pay them much. Finding them, hiring them, training them and losing them so you have to do it all over again adds costs. And they have performance limits. Which is why increased automation reduces head count.