My XD-40 is the same way, you have to pull the trigger. That always bugs me, but it is what it is.
I point the thing at the wall, which is brick on the outside, and do my thing. By that point I know there is nothing in there that will go bang.
Another idea ... I keep a 5-gallon bucket filled with sand on the floor next to the work bench for the same reason.
Wife would be really pissed about the wall
Hopefully you've first removed the magazine & pulled the slide back & then looked in the empty barrel (facing away from you), before pointing the gun at the wall and "doing your thing". If you've done the first three check points, you don't have to worry about something going "bang".