Silverman previously served as president of American Atheists, a non-profit organization supporting the rights of atheists and the removal of expressions of religion in government, from 2010 to 2018. His annual anti-Christmas billboard which calls Christmas "a myth" has often sparked controversy.
During his tenure as Communications Director and Vice President of American Atheists, Silverman made several media appearances, but became especially visible since becoming president.
On April 13, 2018, Silverman was terminated as president of American Atheists over complaints of financial and sexual misconduct. Silverman had been placed on paid leave on April 10 pending an investigation into a complaint received on April 7. The day after his termination, BuzzFeed News published an article detailing allegations of unwanted sexual contact by two women, subsequently denied by Silverman