...and professing themselves wise,they became fools..Bible
It is certainly possible for a person to not believe in God and still behave in a “good” manner, but its sort of like obeying traffic laws but denying that the laws exist.
Except for the likes of Nietzsche, Athiests are walking contradictions.