He's claiming to decry what, exactly? He said that Kaufman is an adviser. He denied that Kaufman is a major player in deciding policy. You can disagree with that answer, but I'm not sure it's that big of a deal.
Can you name me three people who have been closer to Romney in this campaign than Kaufman? Google "Kaufman Romney" and you'll get many hits about their travels together and Kaufman acting as the campaign's chief spokesman and the fact that he is a "senior advisor."