Kennedy chickened out at the Bay of Pigs and refused the air support the invasion needed for its success when the popular support for the invasion didn’t immediately appear. His cowardice invited the soviet missiles.
Kennedy approved the Bay of Pigs invasion but he only approved limited support. In other words Kennedy tried to remove/kill Castro but he only did it in a half-ass manner.
So the question becomes what does somebody like Castro (and the Russians) do when somebody tries to kill them in a half ass manner?
I don't think the answer is that they leave Kennedy alone. That would show weakness on Castro's part.