Microprediction: Building an Open AI Network

by Peter Cotton

How a web-scale network of autonomous micromanagers can challenge the AI revolution and combat the high cost of quantitative business optimization.

The artificial intelligence (AI) revolution is leaving behind small businesses and organizations that cannot afford in-house teams of data scientists. In Microprediction , Peter Cotton examines the repeated quantitative tasks that drive business optimization from the perspectives of economics, statistics, decision making under uncertainty, and privacy concerns. He asks what things currently described as AI are not “microprediction,” whether microprediction is an individual or collective activity, and how we can produce and distribute high-quality microprediction at low cost. The world is missing a public utility, he concludes, while companies are missing an important strategic approach that would enable them to benefit—and also give back.

In an engaging, colloquial style, Cotton argues that market-inspired “superminds” are likely to be very effective compared with other orchestration mechanisms in the domain of microprediction. He presents an ambitious yet practical alternative to the expensive “artisan” data science that currently drains money from firms. Challenging the machine learning revolution and exposing a contradiction at its heart, he offers engineers a new no longer reliant on quantitative experts, they are free to create intelligent applications using general-purpose application programming interfaces (APIs) and libraries. He describes work underway to encourage this approach, one that he says might someday prove to be as valuable to businesses—and society at large—as the internet.

232 pages, Hardcover

Published November 8, 2022

Reviews

[Asat Arslanov · Rating 5 out of 5]

The great playwright and short-story writer Anton Chekhov had hoped for his "brevity is the soul of wit" to be put to good use. The dictum is even more applicable to a mathematically inclined subject such as machine learning, and although the laconic strength of mathematical symbolism would have allowed for an easier task, the author has written the book on AI mostly in the compressed vernacular - to "influence the perception of quantitative work and its future" for a much wider audience. As for a quant that runs aground on the technical clumsiness of the underlying data pipelines world of today, the brilliant exploration of fascinating novel ideas is a breath of fresh air: from statistical crowd-sourcing to interplay between software-engineered oracles and reinforcement learning to temporal difference.

Yes, I like to think of the book more as a play where its main characters "hungry micromanagers" increasingly populate the web of predictions on nano levels....and as an infinite number of infinitesimals can be summed up to calculate an integral, the author argues for clever utilisation of prediction efforts governed through micromanagement (pun intended but no spoilers here!) to build an open-source prediction web where one of the outcomes is bespoke AI on tap. Howzat?! My initial impression was that of new metamathematics for machine learning algorithms - structure-wise somewhat akin to G\''odel numberings.

Moreover, the author generously brings a reader to the "greenfield" of opportunities to actively get involved in the cutting edge AI as deep as one can get at, where the usual suspects are paper, pencil and Python.

[Sam Scott · Rating 3 out of 5]

Interesting, needed better structuring. There is a near constant referencing of other chapters, meaning you never get very far before a train of thought is broken.

As for the prediction web, I think there probably is some critical mass where there is such an abundance of predictions that navigating and reconfiguring this web becomes high value. I think it's flaw is that it operates in a bit of a middle ground.

For problems where predictive accuracy is paramount, or where there needs to be some sort of observability of the systems decision making, a custom solution would probably fit better. Whereas for the other end (the selling of hotdogs, or predictions of mosquito bites), I can imagine a more generalised system of predictions will probably emerge. This might look like something akin to an LLM. Here you would still get "good enough" predictive accuracy, without needing to rely on network effects/ scaling laws.

[M Arteaga · Rating 3 out of 5]

Overall, I found the book challenging due to my limited background in computer program-ming, yet appreciated Cotton's use of memorable examples to simplify technical concepts. I do question the book's premise of the necessity and benefits of maximizing everything through microprediction. Specifically, I ponder about the potential negative implications of a deterministic world governed by micropre-diction, such as a lack of innovation or life becoming mundane.
While recognizing the potential monetary gains from faster, automated decision-making tools, would these benefits diminish over time from a societal perspective?

[PF Chang · Rating 2 out of 5]

The author might as well have an interesting idea, but he never commits to it and discussed it in depth. The whole book reads like a 200 page brochure or a 2 hour long movie trailer.

[William Wilson · Rating 3 out of 5]

The book was definitely written by a mathematician - you are encouraged to skip the chapter on practical applications and instead read about hypothetical irradiated cats.