Public relations and efficiency - public relations to ensure they build up public support which can translate to budget dollars, and to transact routine business. Just like private industry, getting the public to transact business over the Internet, or making non-classified materials available cuts down on the manpower required to handle FOIA requests and such.
I am confident that the systems on the Internet are isolated from the systems that are internal to the organization and do the real work. Data may be passed from the Internet to the internal systems, but generally only in highly structured ways that can be scanned and verified.
This should be the least of your worries - an employee walking out with a disc full of data poses a greater danger.