Robert Lee was, at best, mildly opposed to slavery. He believed it was a necessary evil, and as late as 1865 he was calling it the best situation for blacks in the South.
Having said that I see no reason why his statue should be taken down. You can find flaws with any and all of the men who's statues grace the campus - except maybe the statue of the Longhorn Band Member.
Everyone has a flaw. It's what makes us human.