Find out what social groups or ethnicities the trespasser belongs to.
Make a habit of cleaning your guns and reloading ammo while muttering how much you hate those groups/ethnicity.
Problem solved.
Problem multiplied. The trespasser calls the cops, tells them he fears for his safety, the cops red flag you, and if they don't SWAT you, they will confiscate all your guns, "voluntarily, as a precaution" and you'll play hell ever getting them back.
Remember, the State always knows best.