If the Japanese had only invaded French possessions, it might not have even lifted an eyebrow.
That's my thinking too. However, Japan also needed oil from Indonesia which meant attacking the Free Dutch which may have drawn in the U.S. and British. Then there was the tin in Malasia which would have brought Japan into direct conflict with Britian.