Nate Silver's Blog, page 109

Politics Podcast: The Anatomy Of A Political Scandal

 Subscribe: iTunes |ESPN App |Download |RSS |New to podcasts?

Why do some political scandals stick and others don’t? At what point does a scandal do damage to the politicians involved? Brandon Rottinghaus, a professor of political science at the University of Houston who studies political scandals, joins the FiveThirtyEight Politics podcast to talk about the questions surrounding the Trump administration’s relationship with Russia.

Then, the 2018 midterms are still over a year and a half away, but that doesn’t mean there aren’t elections to watch. Harry Enten shares the latest on the upcoming special elections, and discusses whether they say anything about the electoral direction of the country.

You can listen to the episode by clicking the “play” button above or by downloading it in iTunes, the ESPN App or your favorite podcast platform. If you are new to podcasts, learn how to listen.

The FiveThirtyEight Politics podcast publishes Monday evenings, with occasional special episodes throughout the week. Help new listeners discover the show by leaving us a rating and review on iTunes. Have a comment, question or suggestion for “good polling vs. bad polling”? Get in touch by email, on Twitter or in the comments.

 

How Popular Is Donald Trump?

How We’re Tracking Donald Trump’s Approval Ratings

President Trump’s first real test on the ballot will come 20 months from now, when Republicans face voters at the midterm elections. But those Republicans have decisions to make today about whether to support Trump’s latest policy proposal or criticize his latest tweet. And their fate will be tied to his: Historically, the president’s approval ratings have been one of the best indicators of how his party will fare in congressional elections.

Therefore, we’ve launched an interactive to track Trump’s job approval and disapproval ratings. Although the topline results are fairly similar to other approval-rating averages such as the ones from Real Clear Politics and Huffington Post Pollster — we currently have Trump’s approval rating at 44 percent and his disapproval rating at 50 percent — our version has a few extra features that add a bit of rigor and make it uniquely FiveThirtyEight-ish:

As with our election forecasts, we use almost all polls but weight them based on their methodological standards and historical accuracy.We adjust polls for house effects if they consistently show different results from the polling consensus.And we account for uncertainty, estimating the fairly wide range within which Trump’s approval ratings could vary over the next 100 days. (This is what’s indicated by the green and orange bands on the chart.)

Here’s a more detailed — and at times, technical — rundown of how we crunch the numbers.

Finding and weighting polls

Our philosophy is to use all the polls we can find — provided that we think they’re real, scientific surveys. However, we use a formula that weights polls according to our pollster ratings, which are based on pollsters’ historical accuracy in forecasting elections since 1998 and a pair of easily measurable methodological tests:

Whether the pollster participates in professional initiatives that seek to increase disclosure and enforce industry best practices — the American Association for Public Opinion Research’s transparency initiative, for example.Whether the pollster usually conducts live-caller surveys that call cellphones as well as landlines.

Polls are also weighted based on their sample size, although there are diminishing returns to bigger samples. Surveying 2,000 voters substantially reduces error compared with surveying 400 of them, but surveying 10,000 voters will produce only marginal improvements in accuracy compared with the 2,000-person survey. The worse the pollster’s rating, the quicker it encounters diminishing returns in our formula; a Zogby Interactive survey still wouldn’t get much weight in our model, for example, even if it polled 100,000 people.

The weights also account for how often a pollster measures Trump’s approval ratings. If it does so more often than about once per 20 days, each instance of the poll is discounted so that the pollster doesn’t dominate the average just because it’s so prolific. Daily tracking polls also receive special handling from the formula so that interviews are not double-counted.

“Adults” versus “voters”

Comparisons between different approval-ratings polls aren’t always apples to apples. Some polls are surveying adults regardless of their voting status; others are polling registered voters, and a few are even polling “likely voters.” So far, Trump’s approval ratings are higher among the voter population than among the adult population.

Our default version of the approval ratings reflects a combination of all polls, whether they’re of adults, registered voters or likely voters. If a pollster releases multiple versions of the same survey, however, we use the all-adult version of the poll before the registered-voter version (and the registered-voter version before the likely-voter version). Approval ratings have traditionally been taken among all adults, so this provides for better continuity between Trump’s ratings and those of past presidents.

However, we have another version of the approval ratings that includes only registered or likely voter polls and discards all-adult polls; this may be the most useful version for forecasting Trump’s impact on the 2018 midterm elections. (We also have a version that uses only the all-adult polls and discards any polls that restrict the sample to registered or likely voters, which might be useful for historical comparisons.)

VIDEO: President Trump has record disapproval ratings Calculating a trend line (local polynomial regression)

Because individual polls can be noisy, we estimate how Trump’s approval rating has changed over time using local polynomial regression. Basically, this consists of drawing a smooth curve over the data; this method is similar to those used on Huffington Post Pollster and other sites. In the regression, polls are weighted on the basis that I described earlier, so higher-quality polls with larger sample sizes have more say in the estimate.

While local polynomial regression is a flexible and fairly intuitive method, it’s a bit trickier to work with than it might seem. That’s because people don’t always take the time to determine the correct degree of smoothing, which is governed by several parameters, including the bandwidth and the degree of the polynomial. Too little smoothing can make the curve jut up and down unnecessarily and will result in overfitting of the data. If you smooth too much, however, the curve may be aesthetically pleasing but won’t do all that good a job of describing the data and may be slow to catch up to new trends. While there are usually a wide range of “reasonable” settings when choosing trend-line parameters, our experience has been that people often over-smooth the data when applying these techniques.

For our election forecasts, we choose the degree of smoothing based on what will maximize predictive power. Generally, this results in a fairly aggressive setting, especially in the days and weeks just before an election. This was one of the reasons our model came closer to the mark than most others in last year’s presidential election; it was aggressive about detecting the substantial tightening in the race that came after FBI Director James Comey’s letter to Congress in late October.

In the case of approval ratings, there’s no election to predict — so we instead choose the settings based on how well they would have predicted a president’s future approval ratings. It asks, for instance, what settings would best have predicted Bill Clinton’s approval ratings in March 1998 based on data through February 1998. The analysis is based on approval-ratings polls since 1945.

This also turns out to produce a relatively aggressive model. A week or two is usually enough to detect a meaningful change in approval ratings, and perhaps sooner than that if several high-quality polls tell a consistent story. See the footnotes for more detail about which settings we use.

One more detail: When you see our estimates of Trump’s approval rating for a given date, they reflect only polls that were available as of that date. For instance, our estimate of Trump’s approval rating for March 15 will reflect only polls that have been released to the public by March 15. If on March 18, a new poll comes out that was conducted on March 15, we don’t go back and re-run the numbers for March 15.

Adjusting for house effects

Polls are adjusted for house effects, which are persistent differences between the poll and the trend line. Rasmussen Reports, for example, has consistently shown much better approval ratings for Trump than other pollsters have, while Gallup’s have been slightly worse. The house effects adjustment counteracts these tendencies. So, a recent Rasmussen Reports poll that showed Trump at 50 percent approval and 50 percent disapproval was adjusted by the model to 45 percent approval and 51 percent disapproval. Meanwhile, a recent Gallup poll that had him at 43 percent approval and 52 percent disapproval was adjusted to 44 percent approval and 50 percent disapproval. After adjusting for house effects, therefore, these polls — which had seemed to be in considerable disagreement with each other — are actually telling a fairly consistent story.

The house effects adjustment is more conservative when a pollster hasn’t released very much data. For instance, if a new firm called PDQ Polling releases a survey showing Trump at a 50 percent approval rating when the trend line has him at 44 percent, the model doesn’t assume that PDQ has a 6-point pro-Trump house effect, because its result could have reflected sampling error rather than methodological differences. Therefore, PDQ’s house effect is discounted: The model might adjust its numbers down by 1 or 2 percentage points, but not by a full 6 points. As a pollster releases more data, however, a larger fraction of its house effect is adjusted for. If, over many months, PDQ’s polls have consistently been 6 percentage points better for Trump than the consensus, the model will eventually deduct 5 or 6 points from Trump’s approval rating in a PDQ poll.

As a technical note, house effects and the trend line are calculated on an iterative basis. First the model calculates a trend line using the unadjusted version of the polls. Then it estimates house effects based on how polls compare to that trend line. Then it goes back and re-calculates the trend line, with polls adjusted for house effects. Then it recalculates the house effects adjustment using the recalculated trend line. It loops through this process several times. This helps the model determine whether an apparent shift in the data reflects a real change in Trump’s trajectory or is an artifact of house effects.

Estimating uncertainty

In our election forecasts, we estimate uncertainty by comparing how close past forecasts would have come to past election outcomes. If the polling average in a certain type of U.S. Senate race historically missed the outcome by an average of 4 percentage points a month before the election, for instance, that would be reflected in our forecast. Thus, our calculation of confidence intervals and probability estimates is empirical, rather than being based on idealized (and possibly overconfident) assumptions about how accurate polls “should” be.

For approval ratings, we don’t have that luxury. That is to say, there’s no national plebiscite in which all Americans go to the ballot and vote up or down on whether they approve of Trump’s job performance. Furthermore, our approval ratings estimate blends different types of populations together (adults, registered voters, likely voters), so it’s not clear what such a plebiscite would look like even in theory. Therefore, there’s not any good way to determine how Trump’s “true” approval rating compares to our estimates.

What we can do, however, is measure how well our approval rating estimate on a given date predicts future approval-ratings polls. These estimates are empirically driven, based on an analysis of approval-ratings data from 1945 through 2017 (for Presidents Truman through Obama).

The shaded area (as in the example below) reflects where we project 90 percent of new approval ratings polls to fall. As of March 2, for example, Trump’s approval rating is about 44 percent, but with a range of roughly plus or minus 5 percentage points. Thus, we’d expect Trump’s approval rating to be between 39 percent and 49 percent in about 90 percent of new polls, with the remaining 10 percent of polls falling outside this range. The width of the bands is determined by the volume of recent polling (more polls make it easier to home in on the average), the degree of disagreement in the polls and the amount of long-term volatility. So far, polls have disagreed more on Trump’s approval rating than they did for Obama, so his range is slightly wider than Obama’s would have been.

We’re projecting a range for Trump’s approval and disapproval ratings over the next 100 days. (Or for the 250 days if you look at the “four years” tab of our interactive.) As you can see, uncertainty increases as you advance further into the future. Also, presidential approval ratings tend to be more volatile early in a president’s term, so you should keep a wide range of possibilities in mind for how Trump’s presidency might progress.

In the long run, presidential approval ratings tend to be somewhat mean-reverting: Good ratings tend to get worse, and bad ones tend to get better. They also tend to worsen, slightly, over the course of a president’s tenure in office. It would be easy to overstate these effects, which are relatively minor over the near-to-medium term. Nonetheless, you’d expect a president’s approval rating to decline more often than not when it’s above 50 percent and to rise more often than not when it’s below 40 percent.

The dashed line in the chart indicates Trump’s projected approval rating over the next 100 days, accounting for this mean reversion. Because Trump’s approval rating is within the 40 percent to 50 percent range now, we wouldn’t expect much effect from mean reversion. That is to say, on the basis of how presidential approval ratings have behaved historically — not considering any circumstances particular to Trump — ratings like the ones Trump has now are about equally likely to rise and to fall. It’s true that most presidents begin with higher approval ratings (and much lower disapproval ratings) than the ones Trump now has and then see them deteriorate. But because Trump’s numbers are already middling, he may avoid the slump that past presidents experienced when they exited their “honeymoon period.”

On A Scale Of Pivot To Game-Change, What Did We Think Of Trump’s Speech?

In this week’s politics chat, we discuss how President Trump’s first address to a joint session of Congress changed the political landscape (if it did). The transcript below has been lightly edited.

 

micah (Micah Cohen, politics editor): Welcome, everyone, to our post-not-really-the-State-of-the-Union politics chat. And a special welcome to Perry for his first chat!

Perry, just fyi, these chats are where we don’t use any data and just speculate wildly.

perry (Perry Bacon Jr., senior writer): I have read them.

micah: lol … well-played.

harry (Harry Enten, senior political writer): Welcome, Perry! I’d also like to thank my other colleagues for joining me on this glorious day. Thank you.

perry: I’m excited to be on Team FiveThirtyEight.

clare.malone (Clare Malone, senior political writer): The chats are not team sports, though, Perry. This is every (wo)man for himself.

But we’re really glad to have you in the gladiatorial ring with us, speculating wildly.

natesilver (Nate Silver, editor in chief): Just make sure not to form an alliance with Micah. He’ll double-cross you.

micah: OK, so we’re going to talk about how Trump’s speech on Tuesday night sets up the politics/policy fights in the future. So this chat will be mostly forward-looking, but let’s start with your topline thoughts on the speech itself. Go!

clare.malone: Topline thought is that Trump gave a speech that laid out a lot of ideas for legislation that we have yet to see plans for, which is interesting — and it’s a product of a campaign that was short on actual policy discussion because of the wild optics of the whole affair.

harry: The majority of Americans who watched had a “very positive” reaction to the speech, according to a CNN insta-poll. That’s not unexpected — people who support the president’s party are more likely to watch State of the Union speeches. Of course, as our colleague Ben Casselman noted this morning, Trump was fairly radical in his policy positions, even if not in his tone.

natesilver: I didn’t see the speech live — I watched it after the fact, after I’d read a lot of reactions to it, so that might color my impression. But I thought it was an effective moment for Trump, and I’ve been sort of amused by commentators who have been trying to deny that it was an effective speech. It may or may not be a sign of a “pivot,” but one has to give credit where it’s due if one wants a more “normal” presidency.

perry: On policy, I recommend Ben’s piece. Trump didn’t offer a lot of new ideas. But some of the old ideas that he highlighted — from his campaign and inaugural address — are pretty big in their own right, from repealing Obamacare to the travel ban. On politics, the speech made me start thinking (begin wild speculation here) that he could win a second term if he speaks and acts like he did on Tuesday. He sounded so much more like a normal president than usual.

harry: This was a “pivot” moment. But it doesn’t mean that Trump is pivoting. This is merely a moment in time. The question is whether Trump likes being treated well by the press and decides to copy this behavior going forward.

clare.malone: Oh, Jesus, a “pivot” chat. How many of these have we had?

micah: I don’t think we ever actually fell into the trap of declaring something a pivot, though.

clare.malone: Here’s what I’ll say: Trump’s presidency is not just lived at podiums as other presidencies have been.

micah: We’ve mostly been critical of other people for declaring pivots.

clare.malone: What he did on Tuesday night was certainly calm and deliberate in the fashion that we expect from politicians, but he also exists as a presence on the internet — will the Twitter claims stop, for example? So, I think I will wait to see what he does on that front before I call it a pivot.

natesilver: Here’s the thing, though. Half of the pivots aren’t really pivots, but are a result of the conventional wisdom having become overextended and then having to be pulled back.

micah: But I think Perry’s right that if Trump behaved like he did on Tuesday night for an extended period of time, that could have a huge effect on his fortunes and the political world. It just seems like the odds of that happening are infinitesimal.

natesilver: Sure. But the zeitgeist — or, well, the people in my Twitter feed anyway — have pretty consistently overestimated the degree of political turmoil that Trump was in and underrated his skills as a politician.

micah: So you’re saying that I was right when I alone argued in a chat a couple of weeks ago that it was too soon to declare his administration in disarray?

clare.malone: No.

harry: Someone is fishing for a compliment. His name is Micah Cohen.

micah: I’m fishing for an “I was wrong.”

clare.malone: Wait, because of one speech?

natesilver: I don’t think it was just one speech. The resignation of Michael Flynn and the halting of the executive order on immigration were very serious problems. And they happened in quick succession. But since then, things have calmed down a lot.

micah: Right.

clare.malone: So, in two weeks they’ve solved their problems?

harry: It took two weeks to say the administration was in disarray.

micah: Of course not, but two weeks of problems weigh just as much as two weeks of no problems.

What Harry said.

harry: There’s less “news” coming from the White House. That’s actually a sign that things aren’t as bad as some would have argued.

clare.malone: Well, if the standard is “no one resigned,” then, sure, I guess it’s no longer in as bad a shape.

micah: They clearly still have problems.

clare.malone: OK, but this is supposed to be forward-looking, right?

natesilver: I mean, there are like three separate but overlapping questions here: 1. How much political turmoil is Trump in? 2. How arrayed or disarrayed is the White House, as an organization? 3. How usual or unusual is Trump’s conduct? I think these questions tend to get conflated, especially Nos. 1 and 3.

micah: But Clare’s right — let’s look forward: Even if Trump doesn’t follow through on the tone from Tuesday night and goes back to throwing bombs every other day, does the speech get him anything?

natesilver: It shows that he has multiple gears as a politician. Which I guess we sort of knew already, but Tuesday night was probably the most effective use of “presidential Trump” to date, perhaps along with his rather gracious speech on election night.

perry: What I wonder from Tuesday night is: First, Trump has low approval ratings for a president in his first month; does the speech help move them up? Second, usually a president uses higher poll numbers to push his agenda and insist that Congress bend to his will, because of that support. I’m not totally sure what Trump wants from Capitol Hill. I did not leave Tuesday night thinking that he is desperate for a health care bill to pass the way President Obama was in 2009.

clare.malone: On the approval numbers, I’m pretty open to Nate’s/others’ theory that maybe we’re entering a phase of political life in America where presidents won’t have very high approval ratings because of partisanship. But these coming months will be an interesting test of that vis-à-vis Trump’s legislative agenda, however that ends up taking shape.

harry: Yeah, also, on Perry’s point about Trump’s legislative priorities: We were talking on the live blog about memorable or important joint session and state of the union addresses. I had some trouble thinking of one, and I’d argue that the most important one happened nearly 200 years ago, the one containing the Monroe Doctrine. Usually, they don’t have a big impact. As Bloomberg’s Jonathan Bernstein noted, speeches are really more about sending a signal to Congress. So then the question becomes, “What was the signal Trump sent to Congress?” On the one hand, you could argue that Trump really didn’t give much of a sense of where he wants his partners on Capitol Hill to go. On the other, the tone of his speech perhaps suggests to his party that he’s willing to calm down and actually try to govern.

micah: Agree with Perry that the speech didn’t do much to clarify Trump’s legislative priorities.

clare.malone: He did get specific about getting rid of regulations, though. The Food and Drug Administration drug approval part stuck out to me, in particular, because he wants to get rid of certain testing rounds for drugs, which is a pretty radical change in protections for American consumers.

natesilver: My view is that there’s a fairly wide range for where Trump’s approval ratings could end up in six to 12 months, but that range contains upside and downside cases. Historically, presidential approval ratings tend to revert toward a mean of 45 percent to 50 percent. So expecting Trump’s approval ratings to decline further — as sometimes happens in the first six months of a president’s term — might be wrong since they’re already pretty low.

perry: I left on Tuesday night thinking that maybe Trump’s “signal” was that he will defer to Capitol Hill on a lot of policy issues (tax reform, the Affordable Care Act) but wants them to know he will continue on his Trumpism agenda, much of which involves the executive branch anyway: limiting U.S. involvement in wars abroad; fighting illegal immigration; deportations; the travel ban; his general pro-police, pro-military, anti-Black Lives Matter posture.

natesilver: He did sort of psych-out the media into expecting an immigration pivot when there wasn’t one. Guess you can’t trust “a senior administration official,” even when that senior administration official happens to be Donald Trump!

harry: Two notes: 1. Perry reminds me of what John Kasich was supposedly told, that if he was Trump’s vice president, Trump would be in charge of “making America great again,” while Kasich would be in charge of governing. 2. To Nate’s point: Rarely trust leaks. We should know that by now.

micah: Yeah, and for all the talk of a gentler tone, Trump’s comments on immigration were pretty inflammatory, and that nod to compromise never really materialized.

perry: There was no pivot on immigration, the travel ban, etc. Not to sound too much like Nate, but the media is obsessed with itself. And Trump pivoted by not bashing the media. Hence, his tone was softened, the media declared. It really wasn’t on most issues.

clare.malone: There were shades of his convention speech in that he brought as guests people whose relatives had been killed by immigrants. It was a more highly glossed sheen on “American carnage.”

micah: “American carnage with an ice-cream cone.”

natesilver: Yeah, in some ways, this is Politics 101. Dress up fairly extreme political positions with the auspices of politics as usual. And it’s sort of a wonder why Trump doesn’t try this approach more often.

clare.malone: One might say that it’s not in his nature to be disciplined for any extensive period of time. And this approach requires some wrangling of his natural Trumpian tendencies.

micah: Right. Ben and I were talking this morning, and one thing he said was that this shows that Trump can get a “win” basically whenever he wants by just acting normal for a short period of time.

clare.malone: Were the president not proven to be so capricious a person, I think, of course, the White House would be trying to employ this strategy more often.

natesilver: Well, yeah, that’s the question. We know that Trump’s A game isn’t bad, but how often is he playing his A game?

harry: Let me play a little devil’s advocate: Is Trump’s A game that good? Or is it that his A game is so much better than his normal game that he benefits from low expectations. Put another way: If his A game becomes his regular game, will it still have the same effect?

clare.malone: Lotta words right there, young Harry.

natesilver: I liken Trump to a poker player who is on tilt 80 percent of the time. The 20 percent of the time he’s not on tilt, he’s pretty dangerous, because there’s an added element of unpredictability, even if he isn’t quite doing things by the book. He’s also going to go broke in the long run.

micah: Well, he’s amassed a pretty big pile of chips in the meantime.

natesilver: Oh, let’s not go too far and think Trump is a world-class politician either. I think he’s mediocre and inconsistent, but also somewhat underrated by the people who cover him.

clare.malone: Trump benefits from lowered expectations, but at the end of the day, he doesn’t actually say that much much of the time — his comments on health care reform, for instance, were sort of amazingly empty of substance. And that’s an issue that has long dominated American politics. He’s dangerous when he’s riding a particular wave of the culture well — i.e., attacks on the media. But whether or not that A game on cultural issues translates in the long term into political success remains to be seen. People might get tired of his shtick and just want to see results.

harry: I do worry that we’ll be saying “people will get tired of his shtick” in 2021 when he’s inaugurated for a second term. But what do I know? Maybe they will.

perry: Looking a bit forward, I’m not totally sure where this speech takes us. A normal president would have one big idea in the speech and go campaign around the country for it. I’m not sure what the big idea was in that speech or that Trump cares to talk about one idea in a focused way. The revised travel ban is coming. It sounds like the White House delayed that because they liked how the speech was being covered. But that could quickly get us back to “does Trump’s staff know what it’s doing?” and “is Trump mistreating marginalized groups?” And liberals/Democrats/lawyers/courts will mobilize against Trump.

micah: OK, so that’s a good transition: Let’s switch gears a bit and talk about the Democratic response — not so much former Kentucky Gov. Steve Beshear’s rebuttal as how Democrats generally are responding to Trump’s speech and whether they’re more or less organized than they were for the Neil Gorsuch nomination, for example.

What are Democrats doing, and is it smart?

clare.malone: Democrats are basically acceding that the speech was OK cosmetically — Trump wasn’t raving — but saying that it still contained extremist views of how America should be governed.

harry: On Beshear, these rebuttals rarely matter. They tend to be ridiculed — see Bobby Jindal and the staircase or Marco Rubio and the water — but they aren’t that big of a deal. As such, I thought it was perfectly acceptable to have Beshear give it. I will say it was interesting that they went with a white male from a state that shifted heavily to Trump instead of someone who was black or Latino or from a state that shifted to the left versus the nation as a whole.

natesilver: Only something like 20 percent of Clinton’s voters were white men.

harry: According to the SurveyMonkey exit poll, 25 percent of Clinton voters were white males.

clare.malone: I mean, they wanted to make their frontman someone who had seen the ACA be successful, right?

micah: Yes. Senate Minority Leader Chuck Schumer said as much after the speech.

perry: In the broadest sense, Democrats are doing fairly smart things. They are fighting Obamacare repeal hard, which is important to their base for both substance and political reasons. They are going to fight Gorsuch, but really keeping their grass-roots people more focused on Obamacare, which is more winnable. And they are trying to keep the Russia story going, because that is a place where the media and the Republicans might join them.

micah: So Democrats probably want to move past the State of the Union news cycles pretty quickly?

clare.malone: Well, it seems like it might be swallowed up by another executive order pretty soon, right?

micah: Yeah.

clare.malone: So that’s not necessarily a big worry.

natesilver: Or anything. These speeches don’t usually drive the news for more than a day or two.

clare.malone: Right … we all watched or blogged it, but a lot of people are kinda like, oh, a long speech — I kinda get the idea of what those are like. What’s next? People respond to more kinetic news events than speeches.

perry: Yes. Trump seemed traditional in that speech and said things that maybe were dramatic in policy but were stated in more unifying ways. So I assume that Democrats will wait till Trump goes to his more unorthodox behavior and react to that.

micah: But this does show that Democrats could be screwed if they focus too much on Trump’s tone rather than his policies, right? Because if Trump does ever manage to consistently behave more normally, they’ll be boxed in.

clare.malone: Yes, that ^^^.

harry: Wasn’t that sort of one of the lessons of the 2016 campaign? Clinton focused a lot on Trump’s demeanor. It didn’t work out very well for her.

natesilver: I’m not sure I’d draw too many analogies from Clinton’s campaign. She succeeded in making Trump’s favorability ratings very low. She didn’t give quite enough voters a reason to vote for her, however.

micah: That’s what Harry’s point is, right?

harry: I’m saying Democrats would be right to focus on policy, not demeanor. I’ll add that Trump’s favorability was already low!

natesilver: Yeah, and I think the case isn’t as applicable to what Democrats have to do now. Pure opposition might be a perfectly fine strategy when the other party has all the levers of power. You just have to be smart about your tactics. When Clinton was a presidential candidate, by contrast, pure opposition wasn’t enough and voters needed an alternative.

perry: I think Beshear was trying to link tone to policies. (Trump’s tone is mean about immigrants, and that leads to travel ban.) I think that is hard but more effective. Trump during the campaign was able to suggest that Clinton was dishonest, therefore her policy promises should be ignored.

harry: I concur with Perry.

clare.malone: Well, there is low-hanging fruit with Trump when it comes to things like his “Wall Street Cabinet,” which is something Beshear mentioned. It’s getting voters to read the dissonance of Trump’s rhetoric and then his choices of personnel or policy, as you may spin it.

harry: I guess what I’m saying here is Democrats should, as Perry and Clare have pointed out, try to make the connection between policy and attitude. They could argue that his attitude is as dismissive of others as his policies are. If you don’t make that connection, then Trump seems more presidential, and his policy may seem more normal too.

micah: To close: Final thoughts? Biggest questions?

perry: My biggest question after the speech is how Republicans react. The Republicans on my Twitter feed liked him on Tuesday night. Rep. Darrell Issa said recently that there should be a Russia investigation. (He’s backtracked a bit on that since.) Issa is not Susan Collins. Trump needs Republicans to stop publicly saying things like that. I wonder if the speech helps make sure his party not only votes for Trump (like on the Cabinet picks) but does not undermine him in other ways.

natesilver: To sum up a bit: The speech was an effective one, but also not that important given the lack of policy substance. The question is whether Trump can repeat that performance going forward.

harry: We’re a month in. It’s nearly two years until the midterms and nearly four years until the next presidential election. This is a small blip on the radar, unless Trump can do what he did on Tuesday night over and over again.

clare.malone: I stand by everything I said above and ever.

Politics Podcast: Where Do The Parties Go From Here?

 Subscribe: iTunes |ESPN App |Download |RSS |New to podcasts?

The FiveThirtyEight Politics podcast welcomes Perry Bacon Jr., who recently joined the staff and will be covering the Trump administration from Washington, D.C. Perry discusses President Trump’s budget proposal and previews Trump’s address to a joint session of Congress. The crew also wraps up the “Party Time” series with Galen Druke and debates which wing of which party has been most upended by recent politics, as well as which wing is best poised to capitalize on the shifts.

You can listen to the episode by clicking the “play” button above or by downloading it in iTunes, the ESPN App or your favorite podcast platform. If you are new to podcasts, learn how to listen.

The FiveThirtyEight Politics podcast publishes Monday evenings, with occasional special episodes throughout the week. Help new listeners discover the show by leaving us a rating and review on iTunes. Have a comment, question or suggestion for “good polling vs. bad polling”? Get in touch by email, on Twitter or in the comments.

Why You Shouldn’t Always Trust The Inside Scoop

This is the eighth article in a series that reviews news coverage of the 2016 general election, explores how Donald Trump won and why his chances were underrated by the most of the American media.

So … this is a little awkward. Here we are, in the midst of a series of articles about the media’s shortcomings in covering the 2016 general election. And President Trump, the winner of that election, has to go out and call the media the “enemy of the American people.” Suffice it to say that I’m not on board with Trump’s position. In fact, I’ve had a couple of friends who — while they’ve mostly agreed with the substance of these articles — have asked whether this is really the best time for media criticism. Doesn’t the industry need more solidarity at a time when it’s under attack?

Well, maybe. But I’m worried about that old adage about those who don’t learn from history being doomed to repeat it. There was never much of a learning process after the election, with media outlets finding various scapegoats for the surprising result — the polls! fake news! Clinton’s strategy in Wisconsin! — instead of examining whether there was some deeper problem with their reporting methods. And now, Trump’s screaming, hyperbolic attacks on the press have the potential to drown out any constructive attempts at figuring out what went wrong.

Take, for example, Trump and the White House’s recent admonitions of reporters for using anonymous sources. “Because they have no sources, they just make them up when there are none,” Trump said at the Conservative Political Action Conference on Friday, to take one of many recent examples. By making what amounts to the worst possible version of the argument — ANONYMOUS SOURCES ARE LIES AND SHOULD NEVER BE USED!!! — Trump and the White House make themselves easy to dismiss. One can point out their rank hypocrisy: For instance, that White House chief of staff Reince Priebus recently used unnamed sources to criticize a New York Times article that used unnamed sources, or that White House press secretary Sean Spicer literally shut the doors on news organizations that were seeking comment from him on the record.

But given their exceptionally widespread use — I can’t remember a time when the political news cycle was more dependent on anonymously sourced stories than it is right now — a more nuanced, cool-headed critique of anonymous sourcing might be useful. For example, one could worry — at a time when trust in media is declining — that the “appearance of anonymous sources in routine government and political stories” could strain a news organization’s “trust with readers,” leading to questions about whether it was “carrying water for someone else’s agenda.” Or one could argue that while anonymous sources are sometimes necessary, they should be used only as a “last resort” and only to provide newsworthy information rather than “just spin or speculation.” At the very least, “a story that hangs entirely on anonymous sourcing should always get special scrutiny.”

Those phrases aren’t mine; they’re taken from The New York Times’s policy on anonymous sources. But like most other news organizations’ positions on the anonymous sourcing, the Times’s stated policy has relatively little to do with how these sources are used in practice. So let me tell a cautionary tale of a case where anonymously sourced information wasn’t very reliable, and may have misled readers and reporters. That was in the frequent references to the campaigns’ internal polls at the Times and other news outlets during last year’s election. On Oct. 30, for example, the Times cited private polling from anonymous strategists who claimed that FBI Director James B. Comey’s letter to Congress two days earlier had not seriously affected Clinton’s position in the race:

And while Mr. Trump has crowed about the 11th-hour twist to the race, the F.B.I. director’s letter about the emails has not yet produced a major shift in private polling, according to Republican and Democratic strategists with access to confidential data, who spoke on the condition of anonymity. Mrs. Clinton’s lead over Mr. Trump appears to have contracted modestly, but not enough to threaten her advantage over all or to make the electoral math less forbidding for Mr. Trump, Republicans and Democrats said.

In this instance, the inside scoop turned out to be totally useless. Public polls, like the ones used in the FiveThirtyEight or RealClearPolitics polling averages, were a lot closer to the mark, by contrast. They did show a significant shift after Comey’s letter, with Clinton’s national lead declining from 6 percentage points to 3 percentage points — the difference between a fairly safe lead and one that left her highly vulnerable in the Electoral College.

The thing is, this failure of internal polls wasn’t unusual. The Clinton campaign’s polls dangerously overestimated their candidate’s standing in the Midwest. Meanwhile, the Trump campaign put their candidate’s chances at 30 percent on the eve of the election — the same odds that FiveThirtyEight’s model gave Trump based on public polling — a comparatively bullish projection but one that still had their candidate as an underdog. In 2012, Mitt Romney’s internal polls were laughably off the mark in states including Colorado and New Hampshire, although by all accounts Barack Obama’s were pretty good. So among the past four presidential campaigns, we have one case where internal polls did well (Obama 2012), one where they were mediocre (Trump 2016) and two where they were utter disasters (Clinton 2016 and Romney 2012) — not a very good track record.

Of course, that’s a small sample size. So let’s open things up a bit. FiveThirtyEight’s pollster ratings database contains polls conducted in the final three weeks of presidential, congressional and gubernatorial campaigns since 1998. The database flags polls put out by campaigns or by Democratic or Republican interest groups. I’m going to focus on polls of U.S. House races since they’re the case when internal polls are released to the public most often.

POLL TYPENO. OF POLLSAVG. BIASAVG. ERRORINCORRECT CALLSDemocratic81D +4.88.033%Republican85R +3.87.530%Nonpartisan1,094D +1.56.715%Nonpartisan, except YouGov737D +0.85.919%Published internal and partisan polls are biased and inaccurate

Polls of individual U.S. House races in final three weeks of the campaign, 1998-2014. Excludes races in which a Democrat and a Republican were not the top two finishers.

Polling individual House races is tough no matter what you do, but the partisan and internal polls have been especially bad. Polls conducted by Democratic candidates or groups have been biased toward the Democratic candidate by almost 5 percentage points on average while missing the final margin in the race by an average of 8 percentage points. Republican polls have had roughly a 4-point Republican bias and an average error of 7.5 percentage points. And these partisan polls have also misidentified the winner of the election almost a third of the time, about twice as often as nonpartisan polls have. The nonpartisan polls haven’t been great, but they’ve been much less biased and somewhat more accurate, especially if you exclude YouGov (which makes up a huge portion of the sample because it surveyed almost all House races in 2014).

There are certainly some ambiguities here. Our database only contains polls that were publicly available in advance of the election. These polls were usually released by the campaigns themselves, presumably to help the campaign receive more favorable press coverage. Perhaps the entire sample of internal polls, and not just those that made their way into the public domain, would show better results. Campaigns such as Clinton’s also have a lot of smart people working for them, and they possess voter-file and other proprietary data that’s potentially a lot more powerful than what public pollsters have. They usually also have larger polling budgets than news organizations and so can afford to take fewer methodological shortcuts. Under perfect conditions, therefore, internal polls would potentially be more accurate than public ones.

But circumstances are rarely perfect. In practice, chasing down internal polls isn’t a good use of a reporter’s time and energy, and readers should treat these polls with extreme prejudice when they encounter them in stories. Consider their potential shortcomings:

Campaigns might not be honest with themselves. Think about any company or organization that you’ve worked for. No doubt it has access to all sorts of information that isn’t public. Would you expect it to provide a more honest and accurate self-assessment than a neutral third party? Well, maybe. It would depend a lot on the company’s culture and the incentives of the people doing the assessment. But there’s a good chance the self-assessment would be biased in various ways, probably toward a rosy view of the company’s performance.Campaigns might not be honest with reporters. Unlike a public pollster, a campaign’s objective when discussing polling is to influence coverage instead of providing the most accurate snapshot of the race. Campaigns are selective about what numbers they whisper to reporters and how they characterize their position. And they’re pretty good at spinning. While we don’t spend a lot of time talking to campaigns at FiveThirtyEight, we do so occasionally, and we’re sometimes surprised that the story we get from the campaigns doesn’t match what we’re reading in other news outlets that characterize the campaigns’ positions. Somebody is getting spun, and nobody thinks it’s them.Reporters might misinterpret the campaign’s information. Even if a campaign gave a reporter the straight scoop, the reporter might misinterpret the data or filter it through the lens of whatever prior beliefs they brought to the story. There’s a hint of this in the New York Times snippet that I clipped above, for instance: The story refers to how Clinton’s lead “appears to have contracted modestly” but then asserts that this wasn’t a big deal because of her strong position in the Electoral College. However, the Times’s view that the Electoral College favored Clinton was badly mistaken; just the opposite was true, which is why even a slight tightening of the national race was a big deal.
Obviously, misinterpreting data can also be a big problem when it comes to publicly available information. But it’s especially problematic for proprietary information because the reporter usually won’t provide enough detail to allow for the reader to check the reporter’s work. The reporters who are most skilled at information gathering — at uncovering leads and developing sources — aren’t always the best at information analysis, especially when it comes to technical subjects such as polling, but if the reporter is the only person in the room and can’t reveal many specifics about their experience, you’re being asked to put a lot of faith in their interpretation of complicated evidence.Inside sources rarely provide a complete vantage point. Reporting based on anonymous sourcing often reflects some sides of the story but not others, and this isn’t clear to readers because of the opaque sourcing. It’s worth asking why was the media was so sure of Clinton’s chances, for instance, when it hadn’t been at all confident about Obama’s odds four years earlier. It may had something to do with an imbalance in reporters’ sources. For instance, who were the Republican strategists who told the Times that the Comey letter was no big deal? Were they Republicans working for the Trump campaign or were they #NeverTrump Republicans? In any event, many of the strategists working for Trump had frosty relationships with the press and may not have seen much benefit from spinning horse-race coverage in their favor anyway. Thus, the same reporting methods that had produced too much equivocation in coverage of the Obama-Romney campaign may have tilted analysis of the 2016 horse race in Clinton’s favor.

So suppose I’ve convinced you never to care about a campaign’s internal polls again, or at least not to do so in races where there’s a fair about of nonpartisan polling available. To what extent does this critique impugn reporting based on anonymous or inside sourcing overall, as opposed to being a one-off quirk specific to poll analysis?

I don’t know. The polling beat is unusual in that public and private information compete on a relatively level playing field. In most other domains, one or the other method is dominant. There’s no RealClearPolitics average for reporting on the intelligence agencies, for example, and there aren’t a lot of inside scoops to be had when reporting the weather forecast. Nevertheless, precisely because there aren’t a lot of opportunities to compare them, that access-driven horse-race reporting has somewhat miserably failed as compared with methods based on aggregating public information is interesting. It suggests that newsrooms might place too much of a premium on obtaining proprietary information. And to be clear, this is more access journalism’s failure than polling aggregation’s success; there have been cases where both methods missed the mark, but very few where insiders had the story right but the public pollsters didn’t.

So while I certainly wouldn’t want any news organization to stop aggressively reporting on Trump’s — or any other president’s — administration, I’d encourage reporters who are doing so to abide by their publications’ own standards. Use anonymous sources for matters of legitimate national importance and not for discussing horse-race strategy. Even if the sources are anonymous, provide more detail to readers rather than less; the greater amount of detail was key in The Wall Street Journal’s and The Washington Post’s reporting on Michael Flynn, which forced him to resign as national security adviser.

And even if you can’t name the sources, you can characterize their vantage points. If you have four anonymous CIA sources alleging something about Trump, do they reflect the consensus of top-level officers at the agency, or are they a dissident group? Those are very different stories, but the reader won’t be able to distinguish one group of “senior intelligence officials” from another without this guidance.

Perhaps most importantly, treat stories based on inside information as requiring an especially large dose of skepticism rather than holding them to privileged status. Oftentimes, reporters at powerful news organizations do exactly the opposite, assuming that information is valuable and accurate because it happens to be proprietary. Stories based on anonymous sources can have a romantic allure that conjures up Bob Woodward getting tips from Deep Throat in a parking garage. Now and then the stories indeed live up to that groundbreaking threshold, but other times they’re just glorified intraoffice gossip. Placing a higher burden of proof on these stories can improve the signal-to-noise ratio and help journalism to fulfill its highest aspirations.

Chat: The Biggest Surprises Of The Trump Presidency So Far

In this week’s politics chat, we look back on the first month of President Trump’s tenure and talk about what we were most surprised by. The transcript below has been lightly edited.

 

micah (Micah Cohen, politics editor): Welcome! Today’s topic: What’s been the most surprising thing about the Trump presidency in its first 30 days? This is pretty straightforward. Someone gives a nomination, we debate it, and then the next person goes, and so on. (I’d also be interested if readers have suggestions — tweet ’em to @538politics.)

To decide who goes first, pick a number between 1 and 10 — closest to the number I’m thinking of wins.

natesilver (Nate Silver, editor in chief): 10.

harry (Harry Enten, senior political writer): 5.

clare.malone (Clare Malone, senior political writer): 4.

micah: The number was 4. Harry was so close.

clare.malone: I go first?

micah: Yes.

clare.malone: Micah, you could be duping us about what the number is. But I’ll still go.

I think the most surprising thing about Trump’s presidency so far is the way he has continued to undermine democratic institutions, even though he won the presidency.

My thoughts on this point are twofold:

His continued talk about votes illegally cast in the election.His tweets about the court system, which I think engaged in some dangerous rhetoric that perhaps I was naive in thinking wouldn’t happen once he was president.

harry: What’s interesting about this (and perhaps for many of these) is that we expected Trump to change. I don’t know why we think he’ll pivot, but he never does. I guess we thought he’d become more presidential?

natesilver: I’m surprised that people are so surprised. That’s my biggest surprise.

clare.malone: You guys are putting words in my mouth.

I never said that I expected for him to change — what I’m saying is that I don’t think I predicted that he would talk about, say, the court system in particular.

I think all the talk in recent weeks about how “some of us” knew to take him literally is too cute by half.

natesilver: Are you subtweeting me, Clare?

clare.malone: I’m doing it to your face, Nate, not subtweeting! A lot of reporters/people were taking seriously and literally what Trump was saying during the campaign, but I think I can still sit here and safely say that not many of us could have imagined the president of the United States subverting the court system.

harry: #subslack

natesilver: I don’t necessarily think there was any guarantee that Trump wasn’t going to pivot. But not pivoting was a distinct possibility.

And there are echoes of almost everything he’s done so far in the campaign.

micah: But I think Clare’s point is the particular way in which he’s not pivoted?

harry: The political science literature says candidates keep their promises.

natesilver: I don’t think that’s the right framework either, though, Harry.

harry: So basically you’re just disagreeing to disagree, Nathaniel?

clare.malone: I don’t think talking about the vote count being rigged was a campaign promise. It was something he DID during the campaign but wasn’t a thing he promised to continue, right?

harry: You’re taking me too literally and not seriously enough.

clare.malone: I’d prefer to stay on the narrow point here — I think it’s most instructive.

harry: My point was that candidates tend to follow through on what they say during the campaign. Promise or not. And Trump made a lot of noise about vote counting, so it shouldn’t be surprising that he keeps doing it.

clare.malone: Yeah, but I think this is different. That can’t be what that study says, right? It’s got to be more about policy than rhetoric. And what I’m talking about here is a specific kind of rhetoric that I think goes beyond “voter fraud,” a line that’s been floating through GOP candidate speeches for years.

micah: What specific kind of rhetoric?

clare.malone: Courts. Votes.

natesilver: I guess what I’m sort of trying to figure out is how far Trump will go to push boundaries, in terms of “acceptable” political conduct and discourse. Is he violating norms more than he did during the campaign? Less so? What’s the trajectory? Has there been any change from the first week of the presidency to the fourth one?

And I think it’s a pretty flat trajectory.

micah: Anti-institutional rhetoric.

clare.malone: Sure, you can look at it that way. In which case, throw in attacks on the press.

micah: I normally side with Clare in these chats, but I think I disagree here. The trajectory seems pretty flat to me too. It’s definitely shocking, but I’m not sure it’s surprising.

clare.malone: I’m fine to be the cheese standing alone.

micah: To be fair, it’s hard for me to remember whether I expected Trump to soften his rhetoric after winning.

harry: I actually tend to think Trump telegraphs himself fairly well.

micah: I remember finally signing onto the “Trump won’t change” line in August or September 2016.

harry: The one point where we thought Trump might change was after his victory speech. It was gracious.

micah: Harry, you were next closest. You’re up.

harry: I’m a bit surprised at the almost total lack of movement on reforming the Affordable Care Act.

micah: That was mine.

natesilver: Health care is one heck of a thing to take on. And maybe it shouldn’t be surprising that the Affordable Care Act became more popular once it became under threat.

micah: Maybe it shouldn’t have been, but I was surprised.

clare.malone: I thought you guys weren’t surprised at anything!

micah: Nate apparently isn’t

harry: This is something the Republican Party has been screaming from the mountaintop for years. Trump campaigned on it. And what has happened? Aside from a budgetary measure passed in January, there’s really been nothing but infighting about different possible measures.

natesilver: I was surprised when Trump won the nomination, Clare!

micah: Let’s please not go back to the election.

natesilver: To some extent, it was that period from mid-2015 to mid-2016 when we learned Everything Was Different. But it’s like people didn’t revise their priors enough.

clare.malone: Well, I think the ACA not being moved on has something to do with the Neil Gorsuch Supreme Court nomination — his nomination meant that there was one campaign promise being fulfilled, and it took up a lot of air in the room and distracted people from a lack of substantive conversation around the ACA.

micah: Yeah, also all the controversies.

clare.malone: All of that makes people forget for a little bit what was promised on the stump, to return to promises.

natesilver: Here’s something I’m a little surprised by, which is somewhat related to the ACA stuff. I thought Trump would work with Republicans to pass some big tax cut by now. And that’s gotten stymied in part because they can’t pass a tax cut without screwing up their repeal and replace path on ACA.

micah: By now? We’re only a month in.

harry: It’s normal for it to take time for Congress to get its act together.

natesilver: I’m surprised that there isn’t really even any talk about one. Because tax cuts can be fairly popular — certainly more so than ACA repeal — and would do a lot to unite Trump and the Republicans in Congress.

harry: All the oxygen is being sucked out by Gorsuch and immigration.

clare.malone: Do we think this has to do with their legislative bandwidth being expended on nominee confirmations and that it’ll happen soon?

micah: Wait, let me broaden that question out, Clare. Do you all think that the lack of movement on the ACA is about Trump’s early stumbles/Cabinet confirmations taking longer than expected and that eventually Congress and the administration will get their act together? Or is the delay of repeal and replace more about the law becoming more popular, and when you actually get down to replacing it, you run into a lot of problems? In other words, is the delay so far more logistical? (Suggesting that it can be ironed out.) Or is it more political/substantive? (Suggesting that it might not happen.)

natesilver: On taxes, again, the issue as I understand it is that they have to do health care first, because of the reconciliation instructions that they used.

micah: That doesn’t answer the question.

natesilver: I think passing big, complex, detailed pieces of legislation requires a high degree of competency, and I’m not sure how much of that there is in the White House.

clare.malone: Well … what’s the replace option? I think we don’t really have a clear idea about it, right? And once they get to the point of telegraphing what the plan is, they’ll probably want to be clear that they want to keep the parts that people like (presumably).

harry: I think the delay is more policy-oriented. Look at the backlash against the Cassidy-Collins replace plan from conservative writers like Philip Klein. Klein called their plan “Obamacare Forever.” I don’t see how you can iron that out.

micah: Maybe it’s both competency and policy.

natesilver: Getting buy-in from conservative intellectuals is part of the White House’s job.

micah: OK, Nate, you’re next.

natesilver: Well, I already mentioned the lack of movement on tax policy. I need another one, I guess?

micah: You do.

harry: Nice try.

micah: Stand by for media criticism in 3… 2…

natesilver: I suppose I’m surprised, but maybe I shouldn’t be, by how much it still feels like the campaign, in terms of the frenzied pace of news coverage.

harry: KNEW IT.

natesilver: Nobody learned anything, I don’t think.

harry: Well what have you learned?

micah: Nate, the media is always frenzied, but I think there’s been a good deal of effort to focus on “what matters.”

natesilver: Oh, I totally disagree. I don’t think people realize how close they are to the subject.

clare.malone: Should we just post Nate’s 170-part series here?

natesilver: And so we’ve had a like 36-hour news cycle on fucking Milo Yiannopoulos. It’s amazing how distracted people get from day to day.

harry: At least it’s no longer the top story on Memeorandum.

micah: So, as of Tuesday afternoon, it’s hard to find that story on The New York Times site. It’s still on the Washington Post homepage, but one among many stories. [Editor’s note: As of Tuesday evening, actually, the Yiannopoulos story was leading the Post’s site and prominently featured at the Times. So, I was right for a couple of hours only.]

clare.malone:

One reason I like print paper: perspective. Milo/CPAC is a small story on A11 below the fold.

— Noreen Malone (@NoreenMalone) February 21, 2017

micah: Sister citation!

Nate, your argument is basically that all of media covers lots of things.

natesilver: I have a long and complicated argument, as my 6,132-part mini-series attests to. But basically I think there’s a huge amount of groupthink and herd behavior in how Trump is covered. And a lot of confirmation bias for narratives that aren’t really backed up by all that much evidence. Those are my fundamental critiques.

micah: Oh, well, those are different critiques than what you originally said.

natesilver: Well, and that people get distracted.

micah: Distracted from what?!

natesilver: I’ll turn around the question like so: How often has the same story held the news cycle for more than 48 hours at a time? The Michael Flynn flap and the immigration executive order. Pretty much it, right? Maybe the Women’s Marches?

harry: Gorsuch got plenty of coverage — but, I mean, we’re a 24-hour news cycle nation. What do you expect? This isn’t PBS NewsHour here. (Love that show.)

clare.malone: OK, we could talk about loss of focus on the Russia story. I think that’s a good one.

micah: Isn’t Flynn part of the Russia story?

clare.malone: Until the Flynn reports, people hadn’t talked about the Russia stuff for two weeks, really.

natesilver: It’s adjacent to it, but the non-Flynn parts of the Russia story have fallen somewhat flat.

micah: OK, I’ll concede that. What have the three most important stories of the Trump presidency so far been? I’ll answer my own question …

clare.malone: Typical editor.

micah: It’s immigration order, Russia, protests. Those three things have gotten a ton of coverage.

harry: In no particular order: 1. Trump immigration executive order. 2. Gorsuch nomination. 3. Flynn resignation. (I’m doing specific things here involving Trump.)

micah: Attention lapsed on Russia for a bit, but otherwise I think the media has done a decent job … while also following the ACA stuff, the tax reform stuff.

OK, I want to stop talking about the media. My turn. (Notice how I made an argument and didn’t give people the chance to respond.)

natesilver: Micah Spicer-Cohen

harry: Commander Micah Spicer-Cohen

micah: My most surprising thing: that Bernhard Langer was at the center of a major political story. And I win.

natesilver: Oh come on.

harry: That’s trash.

micah: You want something more substantive?

natesilver: That’s not even like funny ironic; that’s just a cop-out.

micah: OK, real one …

That Democrats (elected officials) have been so disorganized/sloppy/crappy in their opposition to Trump so far.

harry: Interesting, because I’d argue that I’ve been surprised at the grassroots organizing that is so anti-Trump so early in the administration.

clare.malone: Yeah, but that’s grassroots.

micah: The grassroots has been out in force.

clare.malone: I think that’s an interesting one, Micah.

micah: Better than Langer?

natesilver: I thought the Democrats’ lack of a plan to deal with Gorsuch was surprisingly bad.

clare.malone: Democratic politicians appear to be looking to see in what direction their base goes. They don’t want to be seen as dictating top-down after such an embarrassing loss.

micah: They were bad on Gorsuch. They were all over the place on Trump’s Cabinet nominees. Like, why did Jeff Sessions skate through (relatively speaking)?

harry: Well, they got most of the caucus to vote against these nominees. I wouldn’t call it “skating through.”

micah: But it was pretty inevitable that Democrats would vote “no” en masse against Trump’s Cabinet picks.

natesilver: The fact is that Democrats don’t have very much power right now.

micah: It seemed to me like they made a much bigger push against Betsy DeVos than Sessions.

harry: Was it? We both know there’s been record opposition against Trump’s Cabinet picks.

clare.malone: We’ve talked about this offline, but it was interesting to me that Democrats didn’t push harder on Sessions and just followed the viral video outrage at DeVos. Sessions has more power.

Yeah, Micah ^^^

micah: Yes, Clare ^^^

natesilver: Maybe they made a bigger push against DeVos than Sessions because it was plausible that she could lose the confirmation vote?

clare.malone: Well, I also think that ground had been softened for them by how bad her confirmation hearing was. Also … I think there was some sexism.

I don’t want to get into it here, but I think people were really eager to paint her as a dummy. They pinned that rich, dumb lady narrative on her reaallll quick and with a certain gusto.

micah: Yes. Have Democrats really taken down a Trump nominee yet? Does Andrew Puzder count?

clare.malone: Yeah. OPRAH.

natesilver: Sure, Puzder counts.

micah: Hmmm… Didn’t Republicans take him down because he was too pro immigration on top of having a bunch of other problems, including Oprah?

natesilver: One interesting wrinkle here is that it takes 51 votes to scuttle a Cabinet nominee, and there are 48 Democrats.

clare.malone: #math

natesilver: #winning

micah: Anyway, even beyond the nominees. What’s the Democratic message been?

Anyone?

clare.malone: ummm

micah: Bueller?

clare.malone: “Trump bad”

harry: Anti-Trump.

micah: Right. That didn’t work during the campaign..

clare.malone: It’s … the same message Hillary Clinton had. Gulp.

harry: Yes, but it’s not the campaign anymore. Well, not that campaign anymore.

natesilver: “TRUMP BAD” is not a bad message for the midterms, necessarily

harry: Right. What was the Republican message in 2010? Mostly anti-Obama if I recall.

natesilver: Because in the midterms, you sort of don’t have to present an alternative in the way you do when you have your own presidential candidate. Especially when the other party controls Congress too. The argument (often explicit) is that you provide a check on the president’s power.

micah: Yeah, but Democrats don’t have the same built-in midterm advantages that Republicans have.

natesilver: Mostly false.

micah: I’m not sure a pure opposition message works for them. There are more GOP-leaning districts than Democratic-leaning districts. Democrats have to win Republican-leaning districts to win a majority in the House. Republicans need not win Democratic-leaning districts.

How so?

natesilver: People are formulating lots of “Iron Laws Of The Midterms” based on how they worked in 2010 and 2014, when Obama was president. Democrats had a pretty sensational midterm the last time a Republican was president, in 2006.

harry: I’d say the one thing we learned from this past year was that we need to take into account larger samples when possible in terms of making inferences about how politics works.

micah: I didn’t propose any iron laws.

harry: It could be the case that the Democratic coalition — (i.e., young and non-white) — isn’t built for a midterm because those groups turn out at lower rates in off-years. You could also argue that Clinton doing better among voters with more education could lead to them having a strong turnout in a midterm election.

Look, Micah, if I don’t take what you say and create a straw man, how am I supposed to chat?

clare.malone: I can’t even remember what Micah’s original point here was.

natesilver: In Slack Chat, A Straw Man Argument.

I agree with Clare. What was your actual point, Micah?

micah: Democrats have a bad message.

clare.malone: Still.

natesilver: Again, I’d say mostly false.

micah: Or, an incoherent message.

Or, no unified message.

Or, no message beyond “Trump bad.”

Yeah, that last one.

natesilver: I think their message is … fine, I guess … as far as their prospects for 2018 go. It’s if they wanted to block specific things from happening, they haven’t been very effective at that. Like maybe they’d have had a 20 percent chance of stopping Gorsuch instead of a 10 percent chance if they’d been better organized.

micah: So you agree with me, Nate. They have a suboptimal message in terms of preventing policies they oppose.

natesilver: The goal of the opposition party is to win elections.

clare.malone: And just to be clear, we’re saying their message is “Trump bad”?

micah: That’s the most common message I’ve seen, Clare. And if they couldn’t get to 20 percent instead of 10 percent — that’s not a good message.

clare.malone: I mean, as an opposition party, I suppose it’s easiest to have a defensive message. Or a negative message, more precisely.

harry: Opposition parties are opposition.

micah: Yeah, the best countargument to my point — which none of you have made — is that it’s too early. Democrats need time to coalesce behind new leaders and a new message.

natesilver: If the Democratic base is riled up, maybe that means the message has been pretty successful?

micah: Eh, that’s too easy. Of course they’re riled up. The question is what comes of it.

natesilver: I expect — maybe naively — that Democratic messaging will be fairly good on the ACA.

clare.malone: It’s a marathon, not a sprint

natesilver: The attendance at these congressional town halls is certainly something.

harry: What do we know? We know that there’ve been high-profile Republicans who have decided not to run for higher office. We know that Democratic Rep. Cheri Bustos of Illinois is staying in the House rather than running for governor, despite being in a district that swung toward Trump. We know Democrats are having a fairly easy time recruiting for the Virginia elections coming up later this year. All that, to me, suggests that Democrats have been somewhat successful.

micah: We gotta wrap, but before we do — as another measure of how surprised we’ve been — let’s revisit “The ‘Most Powerful Political Players Of 2017’ Draft Extravaganza!!” As a reminder, in mid-December, we each drafted public officials who we thought would be the most powerful people in the Trump era.

natesilver: Oh no.

micah: Ready …

natesilver: I was hoping we’d run out of time.

clare.malone: omg I need to eat lunch.

micah: Here are the teams.

Micah’s:

Donald TrumpMitch McConnellJohn RobertsBernie SandersKellyanne ConwayHeidi Heitkamp

Harry’s:

Cory BookerAnthony KennedyTom PerezJerry BrownNancy PelosiReince Priebus

Nate’s:

President ObamaPaul RyanRex TillersonIvanka TrumpElizabeth WarrenSteven Mnuchin

Clare’s:

Mike PenceJeff SessionsSteve BannonJohn McCainChuck SchumerJoe Manchin

natesilver: So, Clare’s is by far the best team. Harry’s is incredibly awful, but we knew it at the time. Micah, your team and mine have been busts, let’s face it.

micah: Yeah, Clare’s has definitely fared the best so far.

harry: The hell. Your team is trash, Nate.

natesilver: Cory Booker? Jerry Brown?

clare.malone: Cory Booker went to the airport. That was his moment.

micah: No one picked Puzder.

harry: At least I didn’t pick Obama. I don’t even know who that is. Ivanka Trump has been seeing her clothing line die.

natesilver: Obama was president for almost 20 days this year!

harry: And no one has heard from Steve Mnuchin in like 20 years.

micah: I still think my team is the best, by some distance, but Clare’s has closed the gap. Harry’s and Nate’s are waaaay behind.

harry: Clare’s is good.

clare.malone: I never doubted my team for a moment.

micah: Final thoughts?

natesilver: Ivanka and Jared Kushner quashed an anti-LGBT executive order, according to my sources Blavanka Blump and Mared Mushner.

Politics Podcast: One Month In

 Subscribe: iTunes |ESPN App |Download |RSS |New to podcasts?

The Trump administration is officially one month old. The FiveThirtyEight Politics podcast team reflects on some of the biggest changes President Trump has enacted since taking office, including: new immigration policy, his relationship with the press and shifting political norms. The crew also assesses whether Trump’s historically low job approval rating matters all that much in the current political landscape.

You can listen to the episode by clicking the “play” button above or by downloading it in iTunes, the ESPN App or your favorite podcast platform. If you are new to podcasts, learn how to listen.

The FiveThirtyEight Politics podcast publishes Monday evenings, with occasional special episodes throughout the week. Help new listeners discover the show by leaving us a rating and review on iTunes. Have a comment, question or suggestion for “good polling vs. bad polling”? Get in touch by email, on Twitter or in the comments.

Why Polls Differ On Trump’s Popularity

Here’s what we can say for sure: It’s unprecedented for a president to face so much opposition from the electorate so soon. Recent polls show that anywhere between 43 and 56 percent of Americans disapprove of President Trump’s job performance. Even if you take the low end of that range, Trump’s numbers are much worse than any past president a month into his term.

But beyond that, there’s a lot of seeming disagreement in the polls about exactly how unpopular Trump is — and even whether his disapproval rating exceeds his approval rating at all. Moreover, the differences between Trump’s best surveys and his worst ones span a critical range. Take one group of polls, and the country looks about evenly divided — a lot like it did during the 2016 election, when Trump narrowly lost the popular vote but nonetheless won the Electoral College. Take another group, and his electoral fortunes look much bleaker, with Trump already unpopular enough that the House of Representatives could be in play despite Republicans’ advantages from gerrymandering and the geographic distribution of their voters.

What’s the real story? The differences between the polls aren’t random, or at least they don’t appear to be based on the relatively limited amount of data we have so far. Instead, Trump’s approval ratings are systematically higher in polls of voters — either registered voters or likely voters — than they are in polls of all adults. And they’re systematically higher in polls conducted online or by automated script than they are in polls conducted by live-telephone interviewers. Here’s every approval rating poll that we can find for Trump so far this month:

POLLSTERPOPULATION*METHODAPPROVEDISAPPROVERasmussen ReportsLVAutomated + online55%45%Morning ConsultRVOnline4945ZogbyLVOnline4843Fox NewsRVLive caller4847Emerson CollegeRVAutomated4847Harvard-HarrisRVOnline4852YouGovRVOnline4649IpsosAdultsOnline4650SurveyMonkeyAdultsOnline4653CNN / ORCAdultsLive caller4453YouGovAdultsOnline4347Public Policy PollingVotersAutomated + online4353IBD / TIPPAdultsLive caller4248QuinnipiacRVLive caller4251Pew ResearchRVLive caller4254GallupAdultsLive caller4155CBS NewsAdultsLive caller4048Pew ResearchAdultsLive caller3956Average4550A big spread in Trump’s approval ratings

*LV = likely voters; RV = registered voters.
From polls conducted Feb. 1–19.

Source: RealClearPolitics, Huffington Post Pollster, @Politics_Polls

Trump’s overall average is 45 percent approving and 50 percent disapproving. You could come up with fancier ways of calculating the average, such as by weighting the higher-quality polls more heavily. Indeed, we’ll be releasing our own FiveThirtyEight-style approval ratings averages soon. But the more important difference, as I mentioned, is that the average varies based on the sample population and the type of survey:

SURVEY POPULATIONAPPROVEDISAPPROVEAll adults43%51%Voters (registered or likely)4749—SURVEY METHODLive caller42%52%Online or automated4748Trump’s average approval ratings by type of poll

From polls conducted Feb. 1–19.

Source: RealClearPolitics, Huffington Post Pollster, @Politics_Polls

 

All adults vs. registered voters vs. likely voters

Trump has a fairly poor 43 percent approval rating — and a 51 percent disapproval rating — among polls of all American adults, but he improves to a 47 percent approval rating and a 49 percent disapproval rating among polls that survey registered voters or the narrower group of likely voters. That’s a reasonably big difference. So which polls should you use?

Traditionally, approval rating polls are conducted among all adults, so those are probably better for making historical comparisons. And there’s something to be said for inclusivity if your goal is to assess the extent to which Trump has a mandate with the public. He is, after all, the president of all Americans and not just those who are registered to vote or who do so regularly.

But for forward-looking, predictive purposes — to assess the effect that Trump will have on the midterms, for instance — the voter-based polls are probably more useful.

While there can be good reasons for using polls of voters as opposed to those of all adults, however, I’d be wary of making too much about the difference between registered-voter and likely-voter polls. At this early stage, it’s hard to predict what the likely voter electorate will look like in 2018. Midterm voters are typically older and whiter than registered voters overall, which should help Republicans. But they’re also better-educated, which should help Democrats. Furthermore, the “enthusiasm gap” can vary quite a bit from election to election, although it usually favors the opposition party in the midterms (i.e., Democrats in 2018).

The differences between these various types of polls may also narrow as we collect more data. So far, the only pollsters surveying likely voters are Rasmussen Reports and Zogby, and they aren’t very good pollsters. And the only pollsters we could find releasing numbers among both all adults and registered voters — which provides for the most direct comparison between those groups — are YouGov and Pew Research. YouGov’s poll showed Trump’s approval rating lower among all adults than among registered voters, but his disapproval rating was lower also. That’s a fairly typical pattern: Adults who aren’t registered to vote are often politically disengaged and may have indifferent views toward Trump.

Live-caller vs. automated script vs. online polls

Polls conducted with live-telephone interviewers, such as Gallup’s, show worse numbers for Trump — an average of 42 percent approving and 52 percent disapproving — compared to polls conducted online or by automated telephone calls, which have his numbers almost even at 47 percent approving and 48 percent disapproving.

One theory about this is that the online and automated polls reveal “shy” or “hidden” Trump support, with people more willing to reveal their true feelings about the “politically incorrect” Trump in online or automated polls where they have greater anonymity. It’s a plausible theory, but I’m not sure it’s really supported by the evidence. Trump didn’t overperform his polls overall during the Republican primaries, and while he did so in the general election, the overperformance was concentrated among white voters without college degrees, not the group you’d expect if the “shy Trump” theory is right.

There are also arguments against using online and, especially, automated polls. They don’t have as long a track record of success, even if some of them fared reasonably well in the 2016 general election. Moreover, they may have trouble reaching a representative sample of the population. Automated polls only call landlines, which means they miss the roughly half (!!) of the American population that uses mobile phones only. And most online polls don’t use probability sampling, the traditional means of taking a random sample of the population.

Without getting too far into the debate, we’d probably recommend using a mix of live-caller and the higher-quality online surveys, while being suspicious of automated polls.

A further complication is that there’s a relationship between how a poll is conducted and who it’s surveying. For whatever reason, most live-caller approval ratings polls survey all adults, while most automated or online polls survey registered or likely voters. Thus, there’s some cross-contamination between the problems I’ve mentioned here, and it will take more data to sort it out completely.

Be wary of cherry-picking

In the meantime, be on alert for selective citation of polls that are used to advance a narrative. In his press conference last week, for instance, Trump cited a Rasmussen Reports poll showing him with a 55 percent approval rating — neglecting to mention that no other recent poll shows him above 49 percent approval.

But I’ve seen at least as much cherry-picking from liberal and mainstream reporters. In my Twitter feed last week, for instance, a Pew poll that had Trump at 39 percent approval got a lot more attention than a Fox News survey which had him at 48 percent instead.

In some ways, the pattern reflected the one before November’s election, when reporters and pundits selectively interpreted the evidence and assumed that Hillary Clinton was a much heavier favorite than she really was based on the polls. Trump is not very popular, but he’s also no more unpopular than Barack Obama was for much of his presidency. If his numbers hold where they they are right now — especially among registered voters — Republicans would probably hold their own in 2018, and 2020 would be another highly competitive election.

What’s different, as I mentioned, is Trump’s approval ratings are much worse than what a president typically enjoys at this stage of his term. So the question is whether his ratings will continue to decline or if he steadies the ship, or eventually pivots and sees his approval ratings improve. It’s possible — I’d wager more likely than not if forced to bet — that Trump’s ratings will continue to decline over the next six to 18 months, at which point he’d be in trouble since he’s starting from a low baseline. But while he faces a lot of challenges — mostly of his own making — he sometimes benefits from news coverage that overextends itself and predicts his immediate demise only to have to pull back later, perhaps making him seem more formidable in the process. We learned that lesson the hard way in the primaries, and then we often watched the same feeding-frenzy mentality take hold in the general election. While the news is unfolding at an exceptionally brisk pace, changes to Trump’s popularity ratings are likely to be slower.

Emergency Podcast: Trump’s Ties To Russia

 Subscribe: iTunes |ESPN App |Download |RSS |New to podcasts?

The FiveThirtyEight Politics podcast is breaking the emergency glass. After months of bubbling below the surface, questions about President Trump’s ties to Russia are now at the fore. Trump’s national security advisor, Mike Flynn, resigned on Monday after reportedly lying to members of the administration about a conversation he had with Russia’s ambassador. And on Tuesday, The New York Times published a report that Trump campaign aides were in contact with Russian intelligence throughout the campaign.

The podcast team sifts through what’s “smoke” and what’s “fire” amid all the new revelations. They also discuss whether these scandals are likely to stick, and what’s likely to come next.

You can listen to the episode by clicking the “play” button above or by downloading it in iTunes, the ESPN App or your favorite podcast platform. If you are new to podcasts, learn how to listen.

The FiveThirtyEight Politics podcast publishes Monday evenings, with occasional special episodes throughout the week. Help new listeners discover the show by leaving us a rating and review on iTunes. Have a comment, question or suggestion for “good polling vs. bad polling”? Get in touch by email, on Twitter or in the comments.