Note to anyone who responds to this: I'm not particularly interested in getting into a debate. I'm really just wondering what the standard non-theist response is to this sort of question.
I happen to be a theist, but reject the "uncaused cause" argument, in all its forms, for the simple reason that existense cannot have a beginning. But, what you mean by "existense" is the present material universe, and that is a very important distinction. (For example, if existense has a beginning, then so does God, if God exists.)
I also do not believe God created the material univers ex-nihilo, that is, from nothing. (Is God nothing? Before the universe, did He exist nowhere? Absurd, of course.) The Bible does not teach this, actually, only theologians.
These are one theist's answers, but an atheist might be comfortable with them as well. You may have noticed that God made this universe in such a way that to those who do not think to deeply, it seems to be totally self-contained and capable of explaining everything. There is a reason for this.
Hank