I’m afraid you have a very slight understanding of the nature of the Labour Party in the 1930s if you believe that. Insofar as there was any influence of Moscow on Labour (and remember that British Labour was far less influenced by Marxism than the socialist parties of the European Continent, having its roots far more in the social justice movements of the late 19th century and in noncomformist Christianity), that influence was hostile to Nazism right up to the time of the Molotov-Ribbentrop pact.
British Labour was equally hostile to the British Communist Party, which was obviously far more influenced by Moscow. (You have only to look at the output of the Left Book Club, the British socialist think-tank of the time - it’s almost entirely composed of anti-Nazi and pro-Russian tracts).
Although Mosley himself had briefly joined Labour in his journey from Conservative to leader of the British Union of Fascists, there were no parliamentary defections from Labour to that group.
The pacifist wing of Labour lost control with the defeat of Lansbury as leader. From 1935, under Dalton and Bevin, and prompted by the Spanish Civil War, Labour in the Commons voted consistently for rearmament and against appeasement.
Blablabla Labour called Chamberlain a scaremongering warmongerer for increasing military budget as Hitler was rising to power
Labour calling Neville Chamberlain the appeaser a warmongerer against Germany does not strike me as being opposed to Hitler at all