He was in the speaker’s office and according to reports I saw, already started moving.
The office Pelosi was in wasn’t the official speaker’s office and it seems that is where McCarthy is now moving.
The office formerly known as the ex-speaker’s office (RIP Prince)