In a speech at Archbishop Rummel High School in Metairie, Louisiana, Supreme Court Justice Antonin Scalia made it very clear his feeling that the Constitutional guarantee of Freedom of Religion doesn't mean Freedom from Religion, and that the US government traditionally has and should support "religious principles" over non-religious or secular principles.
Should the government should stay out of supporting religion in general? Scalia said, "To tell you the truth there is no place for that in our constitutional tradition. Where did that come from? To be sure, you can't favor one denomination over another but can't favor religion over non-religion?"
If people want the government to be neutral about the idea of religion itself, they should have a vote about it. "Don't cram it down the throats of an American people that has always honored God on the pretext that the Constitution requires it."
http://www.nola.com/politics/index.ssf/2016/01/government_can_should_support.html
http://hosted.ap.org/dynamic/stories/U/US_SCALIA_RELIGION_SPEECH?SITE=AP&SECTION=HOME&TEMPLATE=DEFAULT&CTIME=2016-01-02-15-30-00
Scalia's speech, of course, is itself a perfect example of why government neutrality about religion is so important. It puts the traditions of a nebulous "American people" above the rights of individual Americans. It assumes that a the government can actually choose a way to "honor God" or "favor religion" that even reflects the beliefs of all theistic citizens, not just the local majority; even when couched in non-denominational phrases, such governmental actions almost always reflect a general Christian theism (including Scalia's own reference to God in the speech as a singular male).
Speaking about religion or in religious terms is one thing. A President (or, heck, a Justice) can express their own religious beliefs, even speak about faith or publicly ask for God's blessing as they personally wish. But that's quite different from acting institutionally for religious purposes, or funding religious organizations to do religious things with taxpayer money. As soon as the government starts doing things in the name of God or on behalf of God or to "honor" God, it's establishing religion, even a vague American Christianity, in a way that clearly contradicts the First Amendment.
That people a century or two ago might not have agreed with that doesn't mean it isn't so, with all due respect to Justice Scalia.

What about the fact that belief in the Abrahamic God is not the only form of religion?
The arrogance and presumption of the Christian right is astonishing.
+James Karaganis That's absolutely a major issue.
And Abrahamic (or even just "Judeo-Christian") terminology is often used as a cover for explicitly Christian assumptions about and invocations of God.
For that matter, once you're at that point, it's deucedly difficult to avoid denominational distinctions; Scalia's own Catholicism was often left out of, or even explicitly denied, by some of those "traditional" Christian invocations of the American past.
May I clarify a nuance in US jurisprudence on the Constitutional religion clauses? The 1st Amendment clauses are limits on govt power vs the individual, not grants of power to govt. There is no US legal basis for govt power to do what you are describing.
In US jurisprudence, religious expression is not to be suppressed by govt, and govt power is not to be used to favor one religious group to the detriment of another religious group.
On the other hand, there is no Constitutional protection for the non-religious individual who wants help from govt to free him/her FROM exposure to ALL religious expressions. A grant of that power is just not in there anywhere. Sorry, but it's not US law.
Some other sovereignties do approach the issue from a "freedom FROM religion" perspective, and see the govt as possessing the power to enforce removal of religious expressions from public life, for example, France, called Laïcité there. It is not, and never has been, the US legal approach. And recent US legal cases have continued this approach.
https://en.wikipedia.org/wiki/La%C3%AFcit%C3%A9
+Anne-Marie Clark I have no interest in being freed from all religious expression: I'm a big boy, I can handle having to see things with which I disagree. I simply do not want the power of the Federal Government conscripted to try and force me to believe. There's little practical difference between requiring a person to convert from their own faith to a different, State-sponsored religion (one which they may actually detest) or forcing an atheist to attend services. The Founders clearly did not want government used for that purpose.
+James Karaganis From that you are quite safe. 🙂
PS +James Karaganis , it's true that local govt bodies can get a little out of hand (e.g., school boards in the South…), but the feds? Nah.
+Anne-Marie Clark The feds are generally pretty good about it (at least in enactment, vs. proposal) in Washington, but have been known to stray quite a bit in the field (esp. on the Defense Department wing of things).