Well, not exactly. There was that little matter of Pan Am 103 where he murdered 270, including 189 Americans.
Khadafi decided it was time to clean up his act shorttly after Saddam swung from the end of a rope.
It might not have been exactly that time. But, after he saw was the US could do and did to Iraq and Afghanistan in the wake of 9/11, he knew he might be next.