There is also the ‘Conservation of Matter’ law to consider.
Every atom of your self, and any time machine you may invent, has existed in this universe and nowhere else.
If you go back to some other time, even a few minutes, that matter already exists and you would be introducing more matter into the universe that was there before. Big Bang.....................
Well, duh! Observing The Big Bang is the #1 destination for every curious entity in the universe. All that mass showing up in exactly the same place and time is The Big Bang!