NSF Awards: 1906802
Through our ongoing NSF AISL-funded research on quantitative reasoning and the news, the NewsHour-Knology team has identified polling as a key opportunity for teaching adults some critical statistical concepts. This video will highlight some of that learning and our interventions.
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Hi everyone,
Meaningful Math is a four-year NSF-funded collaboration between PBS NewsHour and Knology to understand how people make sense of numbers in the news and produce better quantitative news content.
This grant provides a too-rare opportunity to reflect on how we are telling stories and to think critically about ways we do that better. We benefit, but more importantly, the ultimate goal is that our audience benefits, absorbing a more meaningful sense of the data and numbers that surround us every day.
Right now, we're focusing on polls to take advantage of people's fascination with them.
Here are some things we're especially proud of thus far!
On the news production side:
We’ve continued to explore new ways to report and share polling data as we approach the 2022 midterm elections. One area we continue to assess is how to visually depict margin of error!
To further highlight the science and research around what we have learned about the coronavirus, we explore key issues, like what it means to be “fully vaccinated,” the nature of the nation’s COVID testing strategy and more. The idea is that these stories contribute to a more nuanced understanding of the pandemic so that people are equipped with knowledge to help them then make more informed decisions.
We also want to better understand how systemic racism has dictated access and affordability of health care and resources, widening disparities for certain communities in disproportionate ways.
On the research side, since this time last year we’ve
Together, we're currently working on:
We’d love to hear from you!
Patti Parson
Managing Producer, Meaningful Math Co-PI
Hi all. Patti from the NewsHour here. Let us know any q's you have about making polls or indeed making any number-centric content more easily grasped!
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
Hey everybody! I'm Laura Santhanam from the PBS NewsHour, and I'm so excited to hear your insightful comments and questions this week. Really looking forward to it!
Kathryn Kozak
This is really interesting. I am curious if you will be providing classroom materials that teachers of statistics classes can use in their classes to further educate people about polling data?
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Thanks! Given how people generally engage with news, we've been primarily focused on understanding takeaways that don't depend on formal education -- but I for one would be interested in partnering with people who work in formal statistics education to do something like this.
Ekundayo Shittu
Science communication has been shown to be much more important now than ever before. We saw how the botched communication surrounding the early days of the pandemic culminated into apathy to vaccination and non-pharmaceutical interventions. I think the value of science communication trascends the use in political polls as eloquently illustrated here. Perhaps this is an avenue for this project to be extended into -- the healthcare domain. Good stuff! I voted!!
Michelle Perry
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Thanks! We're trying to meet people where they are, and we know polls are something they're paying a lot of attention to right now.
Ambika Silva
I love this!! I spend a lot of time on polls both for population/sample talks as well as in confidence intervals. One of the things my students note at the end of the semester is that they're on the hunt for margins of error when they see polls on the news!! That makes me happy :)
Michelle Perry
Jena Barchas-Lichtenstein
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
Music to our ears!! Thank you so much!
Joan Ferrini-Mundy
University President
Interesting project. Great to see the focus on effective science communication connected directly to current issues (like COVID) and widely used techniques such as polling. Does your research include any assessment of the impact on the journalists' understanding of the concepts, or on the effectiveness of their reporting after they have used these tools (in terms of audience understanding, or even the characteristics of their reporting)?
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Thanks, Joan!
We have journals from two early-career journalists throughout their first year working on the project that speak to this question, and our external evaluators will be looking at one of the training workshops we're planning.
But I think my journalist colleagues can also speak to their experience directly. Patti and Laura?
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
Thanks so much for your questions, Joan. Our early-career journalists joined shortly before the pandemic began and quickly became an invaluable part of our team, offering perspective about how to cover STEM-related issues through fresh eyes, including our polling coverage. One has played a critical role in thinking through how our polling graphics could and should present greater nuance.
Patti Parson
Managing Producer, Meaningful Math Co-PI
Thanks @Ekundayo Shittu. We appreciate your vote. We actually also worked with Knology on a Rapid grant around COVID. I can send you the report on that if you're interested. send your email to me at pparson@newshour.org.
Patti Parson
Managing Producer, Meaningful Math Co-PI
Thanks, @Joan Ferrini-Mundy. The information that Knology has shared from their focus groups and analysis have proven very useful in our day-to-day reporting, helping us be more careful in how we share concepts, numbers, graphics and even to question how we use numbers. As to whether our direct audience benefits, we hope so given the greater clarity!
Tichina Ward-Pratt
Educator
Great project. I appreciate the focus on the trustworthiness and accuracy of polling especially considering the political climate.
What is your gauge of how the public will or has responded to more accurate and informed polling?
Do you have evidence from journalists about the effectiveness of the program and how well journalists were able to understand math concepts and decipher a poll’s value?
Michelle Perry
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Laura and Patti can speak to the journalist POV -- but I would say that the challenge isn't journalists' understanding! The challenge is presenting the information in a way that maximizes audience understanding.
We are doing a series of focus groups and experimental surveys with the public, so we can see directly what presentations are more effective. We're making results available on an ongoing basis via through https://knology.org/article/numbers-in-the-news/
Tichina Ward-Pratt
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
Part of the challenge is distilling as much information as possible to its essence and doing so in a way that is not only visually arresting but also clear and easily understood. That can become more complicated, depending on which medium you want to share your information. For example, a television audience only has a few moments maximum to absorb a segment graphic, whereas someone reading a digital print story conceivably enjoys the luxury of more time with the graphics. These are areas we're continuing to monitor and adapt as new information comes to light.
Michelle Perry
Joi Spencer
Interim Dean and Professor of Mathematics Education
Understanding of how to read data and how to be critical of presentations of data is an issue of deep importance. I think we all are wondering whether democracy can survive without these critical skills.
I am so curious about what you are learning during this process. It seems that you are working mainly with adults. For us educators, what do you see as being major misunderstandings amongst those you work with? Do you show the same poll results in different forms and then gather information about what individuals glean from these different forms? If so, what are the big a-has for your team? What are you seeing and what is one example you might give us about what you have shifted as a result.
Thank you for this important work.
Michelle Perry
Jena Barchas-Lichtenstein
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Thanks for these questions!
In terms of polls in particular, it's helpful to highlight the ranges more than the point estimates. At about 2:40 in the video, you can see a visual representation of the uncertainty inherent in polling -- that's one way to help people understand that while 43% or 44% and 60% sounds like a big difference, it may not actually be a meaningful one in this case!
@Laura, can you speak to some of those kinds of changes on the journalist side?
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Forgot to include in my prior post: Yes, exactly - here's how we show the results in different forms and gather information about responses. For example, here's the nitty-gritty research that helped lead to the kind of visualization I mentioned in my previous post: https://knology.org/article/numbers-in-the-news...
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
We've placed greater emphasis on practicing good analytical hygiene when deciding which statistics to amplify in a print story, in a graphic or on-air. Part of that is taking into account the margin of error, but in-house, we've codified going a step further in charts. There, we're visually representing the margin of error to signal to our audience the degree to which findings may be statistically meaningful. We feel it is a firm but gentle nudge to our audience to think about what the significance of what's in front of them. We are also exploring ways to make survey methodology more prominent and engaging to our general audience.
Daniel McGarvey
I agree with Joi. Many of the projects being shared here (including mine) are looking into novel ways to engage and/or teach students in formal or semi-formal settings. But 'Meaningful Math' really has more to do with a functional democracy. I have an indelible memory from my childhood when one of my uncles was teasing me for studying too much. He told me that none of the math I was working on would actually matter later on. You can probably guess what his political stripes look like now. I do fear that with many adults, the die has already been cast and nothing shy of a deeply personal or traumatic experience can open minds. But I would sure love to be proven wrong on this and at a minimum, the lessons learned through this project should be invaluable for building a data-literate population from K-12 students.
Michelle Perry
Jena Barchas-Lichtenstein
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
I should clarify that we're not working exclusively or even primarily on polls. But it's a great example of one of the areas of application of this work.
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
Those are all fair points. Given the seismic shift in trust in public opinion polling that has taken place over the course of recent election cycles, we felt this would pose a great opportunity to think deeply about why and how the public has lost faith in fundamentals around polling and offer resources and different approaches to help people better understand the numbers they see and why and how those numbers do (or don't) reflect their environments and experiences.
But to Jena's point, this is one slice of what we're doing as we are continuing to work on math concepts tied to the pandemic (understanding the scale of loss, nuance in COVID data, etc.), climate change and more.
There's certainly a wide range of issues from which to choose.
Raffaella Borasi
Very valuable project - I love the focus on better preparing news people to understand and report on statistics; I think it could be extremely impactful!
A question I have, though, is how you plan to entice your target population to engage in learning activities around this.
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Our direct target population is journalists -- and we are hearing a lot of interest. Our theory of change is really that the more journalists who make it easier to draw correct inferences, the better off we are across the board.
Laura Santhanam
Health Reporter & Coordinating Producer for Polling
This is exactly right. So many journalists I know work hard every day to do justice to the stories they report and the audiences with whom they go on to share. But so much of that work is conducted against deadline pressure and resource contraints. The intent and hope of our work is to offer insights and examples of how to report with greater nuance to people who are pressed for time by design and in a way that is engaging for audiences who encounter those well-told stories.
Jamie Bell
Thank you for sharing this project. Your approach to helping viewers understand the strengths and limitations of data presented in the news is timely and important. Does the News Hour share what it is learning from the research with others, e.g. PBS member stations?
Jena Barchas-Lichtenstein
Research Lead in Media; Meaningful Math Co-PI
Such a great question! We are co-creating resources for journalists that we plan to present as various journalism conferences, in addition to making them publicly available!
Patti Parson
Managing Producer, Meaningful Math Co-PI
Hi @JamieBell In addition to the resources that Jena mentioned, we always share our best knowledge with PBS stations as we engage with them, especially around election time. We also have a slack channel with 210 public mediamembers on it.
Patti Parson
Managing Producer, Meaningful Math Co-PI
@RafaellaBorassi In addition to what Jena said, we are planning to work with some journalism schools as well, and have already been asked to present at one.
Ho-Chieh Lin
Wow, what a great idea to connect statistics learning with news media! Thank you for sharing :)
Patti Parson
Managing Producer, Meaningful Math Co-PI
Thanks @Ho-Chieh Lin for stopping by. I really appreciated your video and project as well!
Jessica Hunt
What important and relevant work! I look forward to learning more about your work. -Jessica