Fantasy/Sci Fi: MC finds a crystal, becomes a mage and steals his masters memories

[Moose]

Any help would be appreciated, this has been driving me mad all week!

The book is scifi/fantasy, I think I read it around 2005.

Pretty sure the book starts out with the MC a thief, trying to steal something. I think he's in some sewers, using a light spell to see and it triggers a flashback. The MC is much younger, on a different planet and is searching for crystals. The crystals are shaped like sea urchins, and the more spines on the crystal the more valuable it is.

He finds a nearly perfect one but breaks it (I think in a fight with a bully who wanted to take it?), but a mage is able to restore it. The mage offers the MC lessons in magic as payment for the crystal, and takes the MC on as an apprentice.

MC spends months (years?) learning to cast a basic light spell, then many more learning other spells. Each spell requires knowledge of how the world operates and how the spell with change that - eg a light spell requires knowledge of how light moves, reflects, interacts with eyes, interacts with colours, how its generated, all sorts of stuff.

He grows weary of low level, boring magic and finds his masters secret library, and starts learning a spell to absorb a targets memories. The master stumbles upon him doing it, and MC triggers the spell, incorrectly. The MCs master is too powerful, and instead of getting a ... persona of the master in his head to talk to he gets some vague feelings and a one time ability to rewind time by 6 seconds.

He flees his masters home, and throughout the book there are more flashbacks, about how he absorbed some other people, became a thief and learned other spells / abilities.

At the end of the book MC is being tortured by a demon of some sort. MC is about to die, triggers the spell he took from his master. Torture was all mental and took a few seconds, so spell takes him back early enough he can make a different decision and then escape.

I feel like it's a Piers Anthony book, but I've looked at his bibliography and nothing triggered a reaction. It's not a Xanth book, not enough puns.

I think it's a stand alone book, but not sure.

Thanks for any help you can give!!