After successive presidential contests in which the polling industry and the media have come under harsh scrutiny for their election-year practices, The Associated Press is rolling out updated standards for how reporters should cover polls.
The updated guidelines appear in a new chapter in the AP Stylebook — which forms the backbone of the standards used not just by the AP, but by the majority of news organizations around the country.
“A good pre-election poll can provide solid insight into what voters are thinking. In the heat of a campaign, that’s why they are so often intoxicating for journalists, for campaign staffers and, yes, for candidates, too,” said David Scott, AP’s deputy managing editor for operations. “But the 2016 election was a reminder that polls aren’t perfect. They’re unquestionably a piece of the story, but never the whole story.”
The new Stylebook chapter is the latest effort from pollsters and journalists to improve coverage of survey data. In 2016, the coverage of pre-election polling suggested Hillary Clinton was a heavier favorite in the presidential race than the final surveys should have, especially at the state level.
That means, according to the AP, de-emphasizing the horse-race aspects of election coverage and taking care to write about only high-quality polls.
The goal is “to help journalists report responsibly on public opinion research heading into the U.S. midterm elections,” according to the AP.
The AP and its partners — the wire service consulted the Pew Research Center and NORC at the University of Chicago in developing its standards — is hoping the guidelines will be a clarion call for an entire industry, even as some news sources continue to report horse-race results from low-quality, 2018 election surveys.
"It is more vital than ever for policymakers, journalists and citizens to become better informed consumers of surveys and data," said Dan Gaylin, president and CEO of NORC at the University of Chicago, a nonpartisan research institution.
The AP has long discouraged its journalists from predicating stories an entire story on a pre-election poll, but that’s now a bright-line rule, positioned right at the top of the new chapter: “Poll results that seek to preview the outcome of an election must never be the lead, headline or single subject of any story.”
The new guidelines also seek to modernize the wire service’s standards. For the first time, the AP will accept some polls in which respondents “opt in,” as long as the survey includes a component in which others are randomly selected. In the past, the AP rejected opt-in surveys, though it was one of the first news organizations to commission polls conducted over the internet in which respondents were selected randomly.
Opt-in polls “that use additional variables as part of their weighting schemes have shown more promising results, particularly those that use a probability-based sample that is supplemented and/or combined with other sample sources,” the new Stylebook chapter reads. “Because of the difficulty in assessing such approaches and ongoing research into how well they work to reduce bias, the results from such polls should be published only after careful consideration of the techniques used to ensure the results are truly representative.”
“We’re excited about this much-needed update to our survey standards,” said Emily Swanson, polling editor for the AP. “It maintains our commitment to high-quality polling while also taking into account the changing nature of polling and the latest research on poll methods.”