More on this book
Kindle Notes & Highlights
After about a year, the integrity work began producing a modest but statistically significant usage gain, especially among young and infrequent users. Network effects made the true size of the gain impossible to calculate, but it was likely larger than the fraction of a percent the data scientists could measure.
the gains were real. News Feed integrity work had boosted growth for years in ways the company hadn’t understood.
The good news of this discovery was that Facebook could likely boost engagement by investing in stronger News Feed integrity ranking. The bad news was that, by blocking or watering down past integrity work, the company had likely been polluting its platform to the detriment of both users and itself.
Efforts to quantify the growth benefits of integrity ranking work continued quietly, with additional research crediting it with boosting daily usage among new users by 0.9 percent, a huge win on Facebook’s most hallowed metric. People familiar with the work are hopeful that it will eventually prompt Facebook to adopt stronger integrity measures. After all, Zuckerberg isn’t one to leave usage gains on the table.
Important tidbit here. Facebook wasn't measuring engagement over long enough periods to capture dissatisfaction over long periods of time, so they missed that people stopped using Facebook if the News Feed was filled with crap.
This is a well known problem with short A/B tests, but Facebook didn't appear to figure it out until 2022, causing damage for years because they measured success wrong. The goal for the company and shareholders isn't clicks today, it's revenue, usage, and growth over years. Bad proxy metrics in A/B tests easily can get that wrong.
