Apparently, they had planned to attack both but called off the third attack with included the depots.
I think that the excuse was that the Japs knew that they had lost the element of surprise and that the risk would have been much greater.
The third attack was cancelled because ADM Nagumo, was concerned about reports of American submarines possibly in the area of his carriers.