You have hit the nail on the head. A cheap Linux box with a crappy HP laser printer is exactly what my mother in law needs. Then, she can click on anything without worrying about screwing up the computer. One printer cartridge lasts for years.
She surfs the internet and sends email. Linux is excellent for that purpose. She can even use Open Office writer if she ever decides to type something.
The only hard part is finding a dial-up modem that works with Ubuntu.