I doubt Libby was concerned about Wilson's loony rantings at all- it's just that reporters seem to have kept asking various people about Wilson during conversations about more important- and unrelated- matters.
This is from the "Washington Post": "Miller described a meeting on June 23, 2003, with Libby in the Old Executive Office Building. . . Libby was angry about reports that Cheney and other senior officials had embraced flimsy evidence about Iraq seeking nuclear material in the African nation of Niger and was concerned about "selective leaking" by the CIA to distance the agency if no illegal Iraqi weapons were found. Libby noted that the CIA had sent a "clandestine guy" -- a reference to Wilson -- to Niger on a fact-finding mission."
And,
"Libby was concerned about CIA leaks and Wilson's part in it."
In another on line publication, "The Daily Kos," the following:
"He (Libby) was concerned about leaks from the CIA and the CIA not being on board with the administration. Libby claims the CIA, not Cheney, sent Wilson to Niger, and it's the first time Judith (Miller) claims she heard the name Wilson. Libby mentioned Wilson's wife might work for the CIA."
"Wilson's op-ed piece in the NYT appeared on July 6, 2003."
In the Turkishpress.com, this was revealed: "He (Libby) added that the CIA "took it upon itself to try and figure out more" about the uranium allegations without informing either the White House or its own director, the journalist (Miller) recalled."
It is interesting to note that Wilson's op-ed piece about a trip to Niger, taken February 2002, appeared almost a month (July 6, 2003) after the June 23rd interview between Libby and Miller. This says to me that concern was being raised about Wilson, his wife and the CIA before the June 23rd interview and before Wilson's op-ed piece.