I remember reading not too long ago about a few states having laws not allowing those to hold state office positions without "admitting the existence of a higher being." This went to court and ended up being thrown out as being unconstitutional, however it's still on the state books.
Edited:
Texas is a particularly interesting example. The wording is humorous.
In Article 1, Section 4, the Texas Constitution states: "No religious test shall ever be required as a qualification to any office, or public trust, in this State; nor shall any one be excluded from holding office on account of his religious sentiments, provided he acknowledge the existence of a Supreme Being."