David continues in the same psalm to criticize an atheist's moral corruption, and "vile" deeds. This is not surprising, since without an unchanging, objective anchor for ethics, morality just drifts on the changing tides of personal feelings and peer pressure.
There are moral atheists, but they are riding as stowaways on religion's ship. The Nazis were pagans, and we see what kind of society they produced. The Soviets and Chinese Communists were/are atheists, and their governments have murdered more people than Hitler did.
Atoms are all that atheism has to offer, but atoms can't generate morals. Atoms don't think, speak, or make choices. To say that it's important to be good, but then say that everything is atoms, and there's no universal, unchanging standard of good, is foolish. It's irrational.
It's also foolish to disbelieve something just because you don't like it.
There are a lot of things I really don't like, but I know it's dangerous to build my view of reality around my feelings. Does reality ever obey our feelings? People keep smashing themselves to bits on the jagged rocks of reality, because of this foolish way of thinking.
Paul, in Romans chapter 1, says how people reject God, not because they rationally thought it all out, but because they don't like Him. They balk against God because they don't like feeling guilty. So, by not believing in God, they feel freer, and maybe not so guilty.
But would it really a good thing for our society to have millions of people walking around who never feeling guilty? Do you want to live in a community where everyone thinks morality is whatever their own private opinion happens to be at the moment, and they never feel guilty? Does that sound safe?
Some people balk against God because His existence contradicts a desire they feel to imagine themselves self-sufficient. But no one is self-sufficient. Who do you know that is actually self-sufficient? (the answer: no one).
There are people who deny God's existence because something evil happened, whether to them or someone else. But (a) without a moral God, nothing is evil so, given that, how can we complain? (b) What about all the good things that happen? And (c) for some reason we think we deserve to have good things happen. What right to we have to believe that? Maybe we're bad people, and we're lucky things aren't worse than they are.
To believe something because of emotion is foolish.
David wasn't trying to be obnoxious, he was giving a warning. Atheism shuts down your mind. Maybe this is why atheists make so much noise about how scientific and rational they are -- it's loud compensation for being anti-rational and anti-scientific. Among other problems, atheism denies the law of cause and effect, which is the foundation of science.
Atheism claims reason came from mindlessness, matter came out of nothing, complex order came about from millions of random accidents (an event which we never see happen in real life), and social ethics are based on the majority's prejudices in the moment. This is all very foolish, and David doesn't want us to be foolish.