So after a lot of searching I've come to the conclusion (sort of) that this is the book I was looking for. In my mind it was a book series not just one and the humans are never an element. I'm thinking there is a very similar series by a different author, or I never finished the book. There's a chance that my mother was reading the Adept series at the time and I got the 2 mixed in my mind. Any thoughts?