To move from religion to secularism is not so much a loss of faith as a shift into a new set of beliefs and into a new community of faith, one that draws the lines between orthodoxy and heresy in different places.
This is similar to CS Lewis' observation (or was it in JI or Case For God?) that picking and choosing parts of scriptures to believe in make you omnipotent. I need to go back and find that highlight