The Elements of Style, Fourth Edition

by William Strunk Jr.

In dialogue, each speech, even if only a single word, is a paragraph by itself; that is, a new paragraph begins with each change of speaker.