Phrases such as “Goodbye”, “Thank God!”, “Oh God!”, “Oh my God!”, “God speed”, “God forbid”, and “God willing” (even to a degree “Jesus!”) are already secular daily-life phrases but with unsecular origins. Ironically, really religious people tend not to use this type of phrases in any sense at all.
Only an insecure antitheist fascist regime would weed out phrases with unsecular roots in a language. Otherwise, language reflects the history and culture of a society, which makes it fascinating. Any atheist in their good mind would embrace the Christian history of the European culture and not try to erase it.
It is not for nothing that Richard Dawkins insists that the Bible should be taught as a piece of literature in schools.