The Hallucinating Detective

I've put up my 2023 NaNoGenMo project, The Hallucinating Detective . A simulated murder mystery environment is used to coax a variety of large language models into playing detective. Which models will have the guts to actually investigate the mystery, rather than continually saying "Hey, I'm just a text generator, don't ask me!"?

Surprisingly, the answer is—primarily—models trained on something other English prose. Of the twenty-four stories in The Hallucinating Detective, I think the most interesting is "The Adventure of the Rapid-Fire Squirrel". The rift-coder model, trained on Python and Javascript repositories and programming problems, starts off thinking that figuring out whodunit is as simple as writing a solve function, but then starts interrogating suspects and exploring the imaginary crime scene. Many of the other models can't get past asking the suspects why they just said whatever (randomly selected) thing they just said.

I tried to vary up the prose styles in the prompts, from hard-boiled to comedic, but rarely did a model pick up on the request. I will say that em_german_mistral—a model presumably trained on and designed to output German text—not only produced very good English, but picked up that Sherlock Holmes could be a character in a Sherlock Holmes-style mystery: see "Sherlock Holmes and the Draining Pen Affair".

Finally, I want to highlight a beautiful, melancholy poem I discovered in the random Project Gutenberg selections that drive the dialogue in The Hallucinating Detective: "To His Brother Hsing-Chien, Who was in Tung-Ch'uan", written in the year 815 by Bai Juyi.

You are parted from me by six thousand leagues;

In another world, under another sky.

Of ten letters, nine do not reach;

What can I do to open my sad face?

Thirsty men often dream of drink;

Hungry men often dream of food.

Since Spring came, where do my dreams lodge?

Ere my eyes are closed, I have travelled to Tung-ch'uan.