The colorful mayor of New York City, Ed Koch, used to walk the streets asking his constituents, “How’m I doin’?” 

That’s one way to measure a politician’s job approval. Not particularly scientific, but maybe a better approach to getting a straight answer.  

Survey results are so all over the map these days that politicos and pundits can find public survey data to support almost any issue, position or political narrative out there. 

But in the process, those results defeat the very purpose of survey research: situation awareness.

Trump’s poll numbers are a case in point. As of Wednesday at noon on Real Clear Politics, the last 12 public surveys from mid-August to the beginning of September showed President Donald Trump’s job approval anywhere from +10 to -18. 

That’s a 28-point difference. 

No wonder people get confused. 

Certainly, looking at these two surveys, each with very different data, you would come to two dramatically different conclusions, particularly if a survey lacks information about the demographic composition of the sample. 

But even if you chucked the two surveys at either end of the spectrum as outliers and looked at the remaining 10 surveys, Trump’s job approval range still varies from +1 approve to -14, potentially leading people analyzing these surveys to come to contradictory conclusions.  

Overall, Trump’s Real Clear Politics average on the issue of the economy is at -12 (42 percent-54 percent), but it ranges from -4 percent to -22 percent.

On inflation, Trump is down 20 points (39 percent-59 percent); yet, here again, polls have his inflation issue handling from -9 percent to -28 percent.  So, depending on which poll you read, Trump’s approval/disapproval on key economic issues goes from slight disapproval to overwhelming disapproval. 

On the immigration issue, his approval is neutral. His overall is -1 (48 percent-49 percent). The range in this case is +10 to -12. 

With inconsistent results across many surveys and demographics opaque or simply unavailable, it’s difficult for those familiar with survey methodology to assess the validity of data, much less for someone who is unfamiliar with survey research.

So what’s a policymaker, politician or the media to do, given the confused state of survey results? Especially when the sites or institutions managing the research fail to provide enough data to allow consumers of that information to assess the validity of surveys? Is it any wonder that there is a lack of confidence in survey research?

With Congress back from its August recess and off-year elections just two months away, there will be a large number of surveys on the upcoming elections this November and next year’s congressional races as well as key legislative issues in the coming weeks. 

People need to understand that there will be disparities in methodologies used in these surveys, particularly party identification within the sample, but also ideology, age, education, race and a variety of other demographics.  

Increasingly, surveys are being used to validate narratives rather than for critical understanding. This is becoming a problem with both public and private surveys, as they are driven by a preferred narrative or need for fundraising — not situation awareness. We’ve seen the negative outcomes of using this approach.  

In 2022, the “red wave” media focus was fueled by conservative-leaning polls showing certain races identified as competitive but failing to be close on Election Day. This has also occurred in the private sector when organizations have publicized “happy” numbers to support their policy issues, for both media attention and fundraising, that don’t reflect reality. 

Is there a way to determine whether numbers are being manipulated to validate a narrative? 

Generally, when all the numbers are favorable for the narrative of the survey sponsor, there’s a reasonable chance that the data has been manipulated. Either unfavorable data wasn’t shared or questions weren’t asked to ensure a specific outcome.  

This becomes apparent when a survey report sounds more like a press release than an analysis. Consumers of polling data need to take that reality into account when trying to judge the results of any poll.  

All of which circles us back to the fact that there is a remarkable amount of confusion about how the country feels about major policies, the job that Congress and the White House are doing and how those views will affect voters’ choices — all while both sides try to sell their opposing narratives. 

On the economy, Republicans are touting the second-quarter 3.3 percent economic growth number and low unemployment. They argue that inflation is under control and tariffs are bringing in investments into the country and helping reduce the federal deficit. Conservative pundits will use the outlier poll showing Trump with a +10 overall job approval, mentioned earlier, to back up that narrative.

On the other side, Democrats are selling economic doom and gloom, calling out tariffs for rising prices and the “big beautiful bill” as an existential threat to people’s health care, because of cuts to Medicaid and SNAP, and to democracy itself. Democratic pundits will use the alternative outlier poll with Trump’s -18 point overall job approval to support their narrative.  

The immigration issue is no different. Republicans are claiming an immigration success with the border effectively closed and thousands of undocumented immigrants being deported with a focus on “the worst of the worst.” There is survey data showing Trump with a +10 job approval on the immigration issue to support the GOP narrative. 

The Democratic immigration narrative, on the other hand, condemns what they call the administration’s cruel deportation policies implemented without due process — policies they say the public opposes. They, too, have a poll to back up their claims showing Trump’s job approval on the issue of immigration at -12.

So which poll is right? Whose narrative reflects reality? Survey research is a tool to help us understand how people think and feel about their leaders, their policies and their own priorities.  

When the universe of surveys leads to confusion rather than clarity and knowledge, it’s fair to ask the organizations and institutions that manage the research to pull back the curtain and, at a minimum, share their demographic details in the interests of accuracy and accountability. 

David Winston is the president of The Winston Group and a longtime adviser to congressional Republicans. He previously served as the director of planning for Speaker Newt Gingrich. He advises Fortune 100 companies, foundations and nonprofit organizations on strategic planning and public policy issues, as well as serving as an election analyst for CBS News.