What's a non-linear narrative, and should my story be one?

Once upon a time, there were only stories which were about a single person, and they started at the beginning, proceeded by way of a middle stretch of causally linked events, through to the end. Then someone invented the word "meanwhile", and it became possible to tell a story in which that chain of events was partly formed or changed by what was happening or had happened to someone else and in a different place; the story began to step sideways, so as to draw this new set of causal relationships into the main chain of cause-and-effect.

As readers and writers got more experienced, the possible weaves of different strands became ever more interesting and complicated - think Dickens - and began to involve slipping forwards and backwards, as well as sideways, away from the single series of events about one person, told in the order in which they happened. Finally, throw into a writer's calculations the need, in the age of "Look Inside the Book", to make sure that the first page gives the reader an instant hit of excitement - a strong "promise" that this story will be worth your while, as Andrew Stanton puts it - and the result is that more and more books that tell their stories not in the order in which the events actually happened.

The simplest (though not always successful) result of this trend is the ubiquity of prologues, of which more here, but they don't really change the order of the narrative, just give a bit of separate info at the start. And of course, almost any story will have flashbacks, and the occasional "meanwhile" moment. 

I'm talking about stories where the main story is not narrated in the order in which it would have happened in the "real" life you're conjuring. In what follows, I'm using "story" to mean the events of the overall story in the order they actually happened, and "narrative" to mean the text in the order that the events are told on your pages. And I'm suggesting a few examples, but if as you read another example occurs to you, do put it in the comments. For example:

The narrative starts from the end of the story and moves backwards: Martin Amis' Time's Arrow.
The main narrative proceeds in alternating sections taken from different parts of the story: Joseph O'Connor's Ghost Light.
The main narrative has a "frame story" of before, or after, the events - perhaps in a different voice or from a different point-of-view - and sections of it are interspersed among sections of the main narrative: William Boyd's Restless is of this type, though the "frame" is quite substantial.
A series of separate and maybe conflicting narratives, read together, build an overall story: Iain Pears' An Instance of the Fingerpost, William Faulkner's Absalom, Absalom. Sarah Water's Fingersmith can almost be read as one of these.
There are two main narratives, each whole in itself and with no significant plot interconnections as they go - or at least not till the very end.  This is often called a dual narrative: an example would be Charles' Frazier's Cold Mountain. Another example is A S Byatt's Possession although, interestingly, the Victorian "narrative" is almost never actually narrated, but assembled by the reader from the letters, poems and diaries.
There are two main narratives, with different casts of characters; the stories are connected, but their plots don't. This subset of the dual narrative is usually called a parallel narrative: as well as my own The Mathematics of Love, Tobias Hill's The Love of Stones is a super example. 
There are several narratives, nested inside each other: David Mitchell's Cloud Atlas is the classic example.

In addition, with those last three examples where you have more than one main narrative, each narrative might tell its own in the natural chronology, or break its story up and tell it out of order. Mind you, I think it's a very tricky to do that, and not lose your readers ... or possibly your mind.

So, if you're contemplating doing it this way, what should you be asking yourself? 

It's worth remembering that we are time-bound creatures wired for understanding things as a matter of beginning-middle-end: a "causally related chain of events", as the dictionary definition of narrative puts it. If you're breaking that chain up, then effects don't follow causes in your telling, and nor does an effect lead onwards to the next cause. Instead, the reader must intuit or work out consciously the true causal relationships which form the actual story; you're asking more cognitive heavy lifting - conscious or unconscious - as they "assemble" it.

And if you're working with more than one main narrative you're asking for even more heavy lifting: the reader must press the pause button on one story, moving on from it so as to immerse in the other, and then pause that to immerse in the first ... and still make sense of each in its own right.

But, at the same time as you're asking the reader to "assemble" the actual story by separating it out from the narrative form you're using to deliver it, you're asking them to read the separate sections together, to get whatever more the story is offering by being told this way. The thing is, because we are time-bound creatures, the only good reason for asking this extra work of the reader is that their experience of the story will be richer than if you did things the normal way. In other words, the juxtapositions, echoes, thematic links and overlaps, must all help to explain, illuminate, deepen and enrich the story and the ideas it embodies.

So, what matters is that you are sure that the reader's profit-and-loss will end in profit: the cost of the extra work must not outweigh the riches they'll gain. And extra work is not the only possible cost. There's a very real risk that cutting away from strand A and dumping us in strand B - breaking the chain of cause-and-effect and leaving the broken links loose for the reader to pick up - sacrifices both clarity and energy in the storytelling.

In principle, you could compensate for at least some of loss of energy by the fact that you can "create tension", by cutting away from a strand just when we want more. But if you do it solely to deprive the reader of information that we would have if the narrative were chronological, we'll intuit what you're up to, and feel cheated and manipulated. The thing is, overtly witholding information from the reader doesn't create mystery, it just makes us baffled and therefore bored.

What's more, the more invested you've managed to make us in one strand, the more we will resent the switch; indeed, The Mathematics of Love was rejected by an agent on the grounds that readers always resent one or other strand, and I've had other agents agree that it can be a proble: the book overall has to offer many extra riches, to compensate for the risk of readers being annoyed by its structure.  Instead of chopping a strand off just to avoid having to tell us something, you need to find a natural place to end it - a satisfying, chapter-end sort of pause - so that we don't mind leaving it. You must then re-start the other strand in a way that doesn't confuse or frustrate us, but means we can relax and let ourselves immerse all over again, even though we know we haven't got long. 

So, if you're switching from Strand A to Strand B:

Give us enough help, fast, to anchor us in time and place B without us ever having to figure it out. By all means label the sections with the date, or the name, or whatever, but that's not enough: many readers don't read them - or only as a last resort, when they realise they're lost.
I suggest that the opening sentence of the new section should have at least two things which couldn't possibly belong to section we've just finished: places, names, events, whatever. Then make sure the following sentence or two gives more help.
Give us enough to recall the last time we were in B: not just who and what, but what mattered and matters now, what was urgent and important. Be as light-touch as you can, though: just enough to bring it back up to the top of our memory after the intervening A-section. And remember that you may have written the last bit of B weeks ago, but the reader only read it an hour ago. We only need a little reminder, not slabs of repeated information.
Pay attention to the end of the A section we're just leaving. The more A-ish it was, the better the contrast - and so the cleaner and more intuitve the switch - to the world of B.
Make the voices of the different strands as different as you possibly can; either because you have different narrators for A and B, or by using psychic distance and free indirect style to colour the narrative as strongly (and differently) as possible with viewpoint character B. 
Be very wary of opening the new section of strand B with a line of dialogue. Starting any story or new section with dialogue has the enormous disadvantage that it can't directly anchor us in time or space as a line of narrative can. We don't even know who's speaking until we reach the speech tag, and we know nothing more until after it finishes and the narrative can fill us in. Of course it doesn't take much working out ... but it does take some.
Don't be afraid to Tell . There are many times when it's much better to say "In May, in Berlin, John found it difficult to believe Jane wasn't coming back", than do anything subtle and Show-y about how the view from an apartment (which apartment?) showed the linden trees in leaf, but it wasn't much comfort him (to whom?).