Immigration and Customs Enforcement officers escort an arrestee in an apartment building in the Bronx on March 3, 2015, during a series of early-morning raids. (AP/Richard Drew)

On Feb. 22, websites favorable to President Donald Trump touted a poll result that showed wide support for one of the presidents highest-profile immigration policies -- cracking down on "sanctuary cities."

"SHOCK POLL: 80% OF AMERICANS OPPOSE SANCTUARY CITIES," said a headline in the conspiracy-minded site InfoWars. The poll also got prominent play on the conservative websites NewsMax, BizPacReview. Breitbart, Town Hall and American Thinker, as well as on the Russian websites RT and Sputnik.

One of our readers asked us to investigate. We decided not to put the 80 percent finding to the Truth-O-Meter, because we dont doubt the reliability of the poll itself. But questions phrased differently can show dramatically different results.

All told, the finding is a case study in how poll results can be seized by advocates for one side of an issue -- and how some of the nuances in the questions and the answers can be lost in the hubbub.

About the Harvard-Harris poll

The poll result in question came from a new and relatively little-known poll -- the Harvard-Harris survey -- that initially didnt put its full data documentation online. The shortage of information about the poll caused one contributor to the liberal website Daily Kos to wonder whether someone had "catfished" the media with fake data.

We can confirm that the polling partnership is real -- and legitimate. It is headed by longtime Democratic pollster Mark Penn and Stephen Ansolabehere, a professor of government at Harvard University and the director of its Center for American Political Studies. The poll grew out of courses the two have taught together at Harvard. Penn and Ansolabehere are collaborating on the project with the Harris Poll, a venerable public-opinion pollster.

The poll, conducted online between Feb. 11 and 13, was initially released exclusively to the Capitol Hill newspaper The Hill, and the pollsters posted a detailed presentation of the results here,

A closer look at the questions

The question in the poll that attracted the most attention -- certainly from the political right -- was this one: "Should cities that arrest illegal immigrants for crimes be required to turn them over to immigration authorities?" On this question, 80 percent of respondents said yes.

But as we looked into the question, we noticed a couple things.

One was that, despite the blaring headlines about sanctuary cities, the question being asked didnt actually use the words "sanctuary cities." The other concerned words that the question did use -- "arrest" and "crimes."

To understand why such wording choices matter -- and why they could potentially change the results of the poll -- we first need to review what the term "sanctuary city" actually means.

What is a sanctuary city?

There isnt a federal law defining "sanctuary city." Different jurisdictions that use the term -- and even some that shy away from it -- may have some policies in place that other cities dont, and vice versa. But generally speaking, it means they have policies limiting how much local law enforcement assists federal immigration authorities seeking to apprehend and deport people in the United States illegally.

For instance, San Francisco, Los Angeles, and San Diego do not hold immigrants for immigration officials unless they have violent felonies on their records or current charges, according to the Los Angeles Times.

Supporters of sanctuary cities argue that not questioning people about their immigration status builds trust between police and the community, encouraging residents to report crime and help in prosecutions. If an undocumented immigrant gets arrested for a non-immigration offense, they can still be charged, tried and convicted for that crime, Lena Graber, a special projects attorney for the Immigrant Legal Resource Center, told PolitiFact.

Opponents counter that sanctuary cities attract illegal immigration and undermine enforcement of the law. Any crime by someone in the country illegally is a crime that could have been avoided by having removed that person, Ira Mehlman, a spokesman for the Federation for American Immigration Reform, which seeks to reduce immigration levels, told PolitiFact in November.

The question wording

The specific wording used to ask any poll question is always important. Its not even necessarily a comparison of "good wording" vs."bad wording" -- if a question is asked a certain way, it could nudge respondents to answer in one direction.

In this case, respondents were asked about "crimes." To many, that could bring to mind "violent crimes," especially when paired with the term "arrest."

But the experts we spoke to said the jurisdictions described as sanctuary cities dont simply let murderers, rapists, armed robbers and other people they arrest for violent crimes go free. Not only would federal immigration officials be notified, but the violent crimes they were charged with would be prosecuted.

Instead, where the rubber hits the road with sanctuary cities is with lesser, non-violent offenses, even down to a broken tail light -- or simply any interaction with police, such as an undocumented immigrant becoming a happenstance witness to a crime. These are the cases in which police in sanctuary cities would typically be trained to refrain from asking for immigration status or informing federal immigration officials.

And this nuance is not captured by the question that garnered 80 percent support in the Harvard-Harris poll -- as well as the lions share of headlines.

"Question wording always affects responses," said Steven S. Smith, a political scientist and pollster at Washington University in St. Louis. "Arrest for crimes certainly primes the respondent to think that the person is dangerous and therefore should be a high priority in immigration law enforcement."

Karlyn Bowman, a polling analyst at the American Enterprise Institute, noted that there is little history of past polling on the issue of sanctuary cities. But she did uncover one poll released a few days after the Harvard-Harris poll that seems to back up the idea that different wording can substantially change the result.

The poll, conducted by Quinnipiac University and released on Feb. 23, asked, "Thinking about people who have immigrated to the U.S. illegally, who do you think should be deported: Should no illegal immigrants be deported, only illegal immigrants that have committed serious crimes, only illegal immigrants that have committed any crime, or should all illegal immigrants be deported?"

The results: Only 3 percent said no illegal immigrants should be deported, and 19 percent said all illegal immigrants should be deported. But 53 percent of respondents said deportations should only be done for "serious crimes," compared to 22 percent for "any crime."

Thats not just a plurality -- its a majority. And that tells a different story than the Internet headlines.

The pollsters speak

Both Ansolabehere and Penn responded to our inquiries.

Penn called our question about wording differences "an important one" and agreed that finding the right wording in a case like this one is tricky,

"We think the question is fair and clear as worded to obtain the sentiment on the issue," he said. "Had we said serious crimes, that would have tilted it one way and minor crimes would have tilted it the other way. In either case, it would be a different question about the gradations of policy, and a deep dive would surely find some nuances as with all policy."

He added that the task was made more difficult given his teams failure to find any questions on sanctuary cities that had been road-tested in previous polls.

Ansolabehere added, "If you can think of alternative wordings that capture the policy better, please send them to me."

Continue reading here:
Anatomy of a statistic: Do 80 percent of Americans oppose sanctuary cities? - PolitiFact