No. If you recall, Merle reported back to the Governor that he had killed Michonne on a previous mission. Imagine the Gov's surprise when he encounters Michonne in his house, quite alive and with a katana pointed at his (zombie) daughter's neck -- thus learning that Merle had been lying to him the whole time. The Governor is PISSED at Merle, which is why he's pinning the invasion on Merle.
You're right, I forgot about that. But Merle is indeed the key to getting into the prison and that's why the Gov will allow Merle and Daryl to both escape..........