2017 Predictions: Calibration Results

At the beginning of every year, I make predictions. At the end of every year, I score them. Here are 2014, 2015, and 2016.

And here are the predictions I made for 2017. Strikethrough’d are false. Intact are true. Italicized are getting thrown out because I can’t decide if they’re true or not.

WORLD EVENTS

1. US will not get involved in any new major war with death toll of > 100 US soldiers: 60%

2. North Korea’s government will survive the year without large civil war/revolt: 95%

3. No terrorist attack in the USA will kill > 100 people: 90%

4. …in any First World country: 80%

5. Assad will remain President of Syria: 80%

6. Israel will not get in a large-scale war (ie >100 Israeli deaths) with any Arab state: 90%

7. No major intifada in Israel this year (ie > 250 Israeli deaths, but not in Cast Lead style war): 80%

8. No interesting progress with Gaza or peace negotiations in general this year: 90%

9. No Cast Lead style bombing/invasion of Gaza this year: 90%

10. Situation in Israel looks more worse than better: 70%

11. Syria’s civil war will not end this year: 60%

12. ISIS will control less territory than it does right now: 90%

13. ISIS will not continue to exist as a state entity in Iraq/Syria: 50%

14. No major civil war in Middle Eastern country not currently experiencing a major civil war: 90%

15. Libya to remain a mess: 80%

16. Ukraine will neither break into all-out war or get neatly resolved: 80%

17. No major revolt (greater than or equal to Tiananmen Square) against Chinese Communist Party: 95%

18. No major war in Asia (with >100 Chinese, Japanese, South Korean, and American deaths combined) over tiny stupid islands: 99%

19. No exchange of fire over tiny stupid islands: 90%

20. No announcement of genetically engineered human baby or credible plan for such: 90%

21. EMDrive is launched into space and testing is successfully begun: 70%

22. A significant number of skeptics will not become convinced EMDrive works: 80%

23. A significant number of believers will not become convinced EMDrive doesn’t work: 60%

24. No major earthquake (>100 deaths) in US: 99%

25. No major earthquake (>10000 deaths) in the world: 60%

26. Keith Ellison chosen as new DNC chair: 70%

EUROPE

27. No country currently in Euro or EU announces new plan to leave: 80%

28. France does not declare plan to leave EU: 95%

29. Germany does not declare plan to leave EU: 99%

30. No agreement reached on “two-speed EU”: 80%

31. The UK triggers Article 50: 90%

32. Marine Le Pen is not elected President of France: 60%

33. Angela Merkel is re-elected Chancellor of Germany: 60%

34. Theresa May remains PM of Britain: 80%

35. Fewer refugees admitted 2017 than 2016: 95%

ECONOMICS

36. Bitcoin will end the year higher than $1000: 60%

37. Oil will end the year higher than $50 a barrel: 60%

38. …but lower than $60 a barrel: 60%

39. Dow Jones will not fall > 10% this year: 50%

40. Shanghai index will not fall > 10% this year: 50%

TRUMP ADMINISTRATION

41. Donald Trump remains President at the end of 2017: 90%

42. No serious impeachment proceedings are active against Trump: 80%

43. Construction on Mexican border wall (beyond existing barriers) begins: 80%

44. Trump administration does not initiate extra prosecution of Hillary Clinton: 90%

45. US GDP growth lower than in 2016: 60%

46. US unemployment to be higher at end of year than beginning: 60%

47. US does not withdraw from large trade org like WTO or NAFTA: 90%

48. US does not publicly and explicitly disavow One China policy: 95%

49. No race riot killing > 5 people: 95%

50. US lifts at least half of existing sanctions on Russia: 70%

51. Donald Trump’s approval rating at the end of 2017 is lower than fifty percent: 80%

52. …lower than forty percent: 60%

COMMUNITIES

53. SSC will remain active: 95%

54. SSC will get fewer hits than in 2016: 60%

55. At least one SSC post > 100,000 hits: 70%

56. I will complete an LW/SSC survey: 80%

57. I will finish a long FAQ this year: 60%

58. Shireroth will remain active: 70%

59. No co-bloggers (with more than 5 posts) on SSC by the end of this year: 80%

60. Less Wrong renaissance attempt will seem less (rather than more) successful by end of this year: 90%

61. > 15,000 Twitter followers by end of this year: 80%

62. I won’t stop using Twitter, Tumblr, or Facebook: 90%

63. I will attend the Bay Area Solstice next year: 90%

64. …some other Solstice: 60%

65. …not the New York Solstice: 60%

WORK

66. I will take the job I am currently expecting to take: 90%

67. …at the time I am expecting to take it, without any delays: 80%

68. I will like the job and plan to continue doing it for a while: 70%

69. I will pass my Boards: 90%

70. I will be involved in at least one published/accepted-to-publish research paper by the end of 2017: 50%

71. I will present a research paper at the regional conference: 80%

72. I will attend the APA national meeting in San Diego: 90%

73. None of my outpatients to be hospitalized for psychiatric reasons during the first half of 2017: 50%

74. None of my outpatients to be involuntarily committed to psych hospital by me during the first half of 2017: 70%

75. None of my outpatients to attempt suicide during the first half of 2017: 90%

76. I will not have scored 95th percentile or above when I get this year’s PRITE scores back: 60%

PERSONAL

77. Amazon will not harass me to get the $40,000 they gave me back: 80%

78. …or at least will not be successful: 90%

79. I will drive cross-country in 2017: 70%

80. I will travel outside the US in 2017: 70%

81. …to Europe: 50%

82. I will not officially break up with any of my current girlfriends: 60%

83. K will spend at least three months total in Michigan this year: 70%

84. I will get at least one new girlfriend: 70%

85. I will not get engaged: 90%

86. I will visit the Bay in May 2017: 60%

87. I will have moved to the Bay Area: 99%

88. I won’t live in Godric’s Hollow for at least two weeks continuous: 70%

89. I won’t live in Volterra for at least two weeks continuous: 70%

90. I won’t live in the Bailey for at least two weeks continuous: 95%

91. I won’t live in some other rationalist group home for at least two weeks continuous: 90%

92. I will be living in a house (incl group house) and not apartment building at the end of 2017: 60%

93. I will still not have gotten my elective surgery: 90%

94. I will not have been hospitalized (excluding ER) for any other reason: 95%

95. I will make my savings target at the end of 2017: 60%

96. I will not be taking any nootropic (except ZMA) daily or near-daily during any 2-month period this year: 90%

97. I won’t publicly and drastically change highest-level political/religious/philosophical positions (eg become a Muslim or Republican): 90%

98. I will not get drunk this year: 80%

99. I get at least one article published on a major site like Huffington Post or Vox or New Statesman or something: 50%

100. I attend at least one wedding this year: 50%

101. Still driving my current car at the end of 2017: 90%

102. Car is not stuck in shop for repairs for >1 day during 2017: 60%

103. I will use Lyft at least once in 2017: 60%

104. I weight > 185 pounds at the end of 2017: 60%

105. I weight < 195 pounds at the end of 2017: 70%

Some justifications for my decisions: I rated the civil war in Syria as basically over, even though Wikipedia says otherwise, since I don’t think there are any remaining credible rebel forces, and ISIS is pretty dead. Trump’s approval rating is taken from this 538 aggregator and is currently estimated at 38.1%. I rated the border wall as not currently under construction, despite articles with titles like The Trump Administration Has Already Started Building The Border Wall, because it was referring to a 30-foot prototype not likely to be included in the wall itself (have I mentioned the media is terrible?). I refused to judge the success of the Less Wrong renaissance attempt, because it seemed unsuccessful but was superseded by a separate much more serious attempt that was successful and I’m not sure how to rate that. I refused to judge whether or not I got a new partner because I am casually dating some people and not sure how to count it. I refused to judge whether I got 95th percentile+ on my PRITE because they stopped clearly reporting percentile scores.

This is the graph of my accuracy for this year:

Of 50% predictions, I got 5 right and 3 wrong, for a score of 62%

Of 60% predictions, I got 14 right and 8 wrong, for a score of 64%

Of 70% predictions, I got 8 right and 5 wrong, for a score of 62%

Of 80% predictions, I got 16 right and 2 wrong, for a score of 89%

Of 90% predictions, I got 24 right and 1 wrong, for a score of 96%

Of 95% predictions, I got 8 right and 1 wrong, for a score of 89%

Of 99% predictions, I got 4 right and 0 wrong, for a score of 100%

Blue is hypothetical perfect calibration, red is my calibration. The multiple crossings of the blue line indicate that I am neither globally overconfident or globally underconfident.

Last year my main concern was that I was underconfident at 70%. I tried to fix that this year by becoming more willing to guess at that level, and ended up a bit overconfident. This year I’ll try somewhere in the middle and hopefully get it right.

There weren’t enough questions to detect patterns of mistakes, but there was a slight tendency for me to think things would go more smoothly than they did. I overestimated the success of my diet, my savings plan, my travel plans my job start date, my long-FAQ-making ability, and my future housing search (this last one led to me spending a few weeks at a friend’s group house, failing on a 95% certainty prediction). I only made one error in favor of personal affairs going better than expected (SSC got more hits than last year; maybe this isn’t a central example of “personal affairs going smoothly”). None of these really caused me any problems, suggesting that I have enough slack in my plans, but apparently I’m not yet able to extend that to being able to make good explicit predictions about.

My other major error was underestimating the state of the US economy, leading to a couple of correlated errors. I think I got Trump mostly right, although I may have overestimated his efficacy (I thought he would have started the border wall by now) and erred in thinking he would lift sanctions on Russia.

Otherwise this is consistent with generally good calibration plus random noise. Next year I’ll have played this game five years in a row, and I’ll average out all my answers for all five years and get a better estimate; for now I’ll just be pretty satisfied.

Predictions for 2018 coming soon.