After three books' worth of setup, we finally get to learn what's actually in the Tower. At least, that's the idea ... but first we have to have the big war between Sam Clemens and King John. Of course they have to meet, which means they have to get pretty far up-River, and both sides have to be somewhat balanced, so we have to read about more people first, which means that we have to read about their stories on Earth and what "actually" happened first, and then we have to re-read any events in which they'd been previously involved on the Riverworld ... yeah. You get the idea.
Farmer's habit of explaining everything in detail really starts to become a problem in this book. Everything we've seen before has to be explained again, every character has to go into detail about what they believe is happening, etc. etc. etc. As a result, it's almost a relief to get to the war, which really shouldn't be the point in a series that values ethical behavior so highly.
On top of that, as you'd expect, a good number of the characters we've met are killed in the war (otherwise you'd have a party of 40-50 people on those narrow ledges and such), which takes a bit out of the story. Finally, most of the trip to the Tower is simply a repeat of the events from the Egyptians and Joe Miller, so even that isn't as interesting as it should have been. (There isn't much drama about getting to the Tower, either. Of course there will be a way. If there weren't, what kind of book would this have been?)
The final section in the Tower itself is interesting ... but even that ends in a very unsatisfying fashion, almost as if Farmer deliberately chopped off the story to prepare for a fifth book. (It also suffers somewhat from the magitech that Farmer had to use: in the late '70s, there would have been nothing even approaching Riverworld technology, and again, even personal technology was quite limited, so much of what the Ethicals have is basically Star Trek-type stuff.)
As compelling as the story was in the first three books, The Magic Labyrinth is disappointing. I remembered this book pretty much all the way through, and I expect I felt the same way when I first read it. Other people may find it a more fitting "end" to the story, but I wanted (view spoiler)[at least some kind of idea about the people still in the Valley, and what might happen next for the people in the Tower. Instead, we get nothing (hide spoiler)]