NSF Awards: 2101669
Middle school science teachers want technology to amplify their impact on the hundreds of students with multiple, diverse ideas in their classes. By exploiting advances in natural language processing (NLP) this project detects the rich ideas that students develop during everyday experience. Aligned with the Knowledge Integration pedagogy, this project takes advantage of NLP methods for detecting student ideas and creates adaptive guidance that gets each student started on reconsidering their own ideas, considering their peers’ ideas, and developing more coherent explanations for complex scientific phenomena. This work continues a successful partnership between University of California, Berkeley, Educational Testing Service (ETS), and science teachers and paraprofessionals from six middle schools enrolling students from diverse racial, ethnic, and linguistic groups whose cultural experiences may be neglected in science instruction. The idea detection technology adapts to each student's cultural and linguistic entry points into the topic. Our video illustrates how TIPS empowers students to use their ideas as a starting point for deepening science understanding.
Catherine McCulloch
Senior Project Director
Hello Marcia and team! This is fascinating work. I would love to know more about so many aspects of it. For instance, would you say more about how this web-based system helps students consider their peers' ideas?
Allison Bradford
Graduate Student Researcher
Hi Catherine! Thank you for your question. As we further develop the NLP techniques that detect the ideas expressed in each student response, we hope to use the technology to dynamically partner students. For example, this would enable us to automatically pair students who express different ideas about a phenomenon and use the Thought Buddy to scaffold a discussion where the students consider each other's ideas. While we our still in early stages of this project, we are leveraging our partner teachers' insights about how to engage students with their peers' ideas. For example, they have suggested having students compare the prompts they received from the Thought Buddy or displaying interesting chat examples to prompt whole class discussion.
Allison Bradford
Graduate Student Researcher
Mike Szydlowski
K-12 Science Coordinator
I love this idea and want to try it! Like any good program or idea, this takes time. What would you say that the teachers gave up to spend more time on this process? Do they feel it was a good trade off?
David Campbell
Program Officer, retired
Hi Marcia and team! I loved hearing the students say they didn't have to worry about "being wrong." It seems the thought buddy removes some of the peer pressure students face in the classroom. Following up on Mike's question, many teachers feel they face conflicting challenges of "covering the material" and taking the time of take advantage of students' individual ideas. I wonder if your teacher participants expressed that concern.
Sarah Bichler
Post-Doctoral Researcher
Hi Mike and David,
thanks for your interest and this important question. Teachers I worked with definitely expressed the concern not being able to cover the material - when trying to let students explore more, bringing in students' ideas, letting students revise, etc. With respect to the dialog, teachers really appreciated that each student was able to discuss with the avatar and that each student got unique feedback, something they could not have done in the time the chat took. I have seen teachers weave in students' ideas to jump-start a class discussion, it did not take long at all :). We hope that the dialog makes many ideas visible, placing them for being taken up beyond the chat and working with teachers to find ways in which students' ideas can become central to instruction and ways in which these ideas can help to "cover the material".
Amy Robertson
This is so cool, y'all! I have so many thoughts bubbling up. As someone who deeply values responsive teaching -- particularly responsive teaching that attends closely to the *substance* of students' thinking -- I've been somewhat skeptical of NLP because it feels like it can miss some of the depth and connections within student ideas. BUT some folks on our team just did an analysis of student responses using SLP and found that with a little bit of additional work on the part of analysts (looking at some of the top example responses), they could identify many of the same patterns that another team of researchers did without NLP. So in this case, NLP was a really helpful tool.
I'm processing out loud a bit, but I'm curious about the trade-offs you see along the lines of being able to discern and respond to substance... And we'd love to chat with you offline sometime. We share a deep value in framing students' ideas as generative and finding ways to build from there.
Sarah Bichler
Post-Doctoral Researcher
Hi Amy, thanks so much for stopping by!
I love these thoughts bubbling up! I feel if the human coding pays attention to the depth and connections within student ideas, the machine is likely to pick it up. We refine our coding multiple times and go through a rigorous training procedure before coding or annotating our training data. Our results show that if the model can detect an idea, it actually does it very well (accurate and is sensitive to the idea). We were more worried about the infrequent ideas, that we don't find ways to make these visible, an area we are still working on.
With respect to your question, I feel there are trade-offs responding to different ideas students express. If a student has multiple ideas, which one do we respond to and why? We don't know yet what's most effective. For example, a student thinks that a metal bowl conducts cold faster than a wood bowl. I see two productive ideas. The student could explore the idea of "cold flow" more, investigate, and develop it. Or, the student could expand and build on the idea of the different rates of thermal conductivity...
I'd love to talk more!
Sarah
Amy Robertson
Ann Podleski
I think this type of tool has a lot of potential for some students. And every different "tool" for engaging and promoting student reflection and thinking is helpful for teachers. The fact that students can "ask anything" and not worry about what the teacher might think is one of the benefits I can see for this. Nothing should totally replace the "human" interaction, but I can see this enhancing learning and teaching and allow for teacher's time to be used in the ways that are most effective for each different student. And I think there are clear trade-offs, but each additional venue we provide is a plus. Thanks for this work!
Sarah Bichler
Post-Doctoral Researcher
Hi Ann,
thanks so much for sharing your thoughts with us! I speak for our entire team when I say I absolutely agree with you that no technology should replace the teacher, nor would it ever be able to do that. We are working closely with teachers to learn how these tools best support them in their practice and to refine the tools we design such that we can maximize their value for practice. The potential of such tools might be that they help us scale what effective teachers do anyways. When teachers are facing many demands on different levels of teaching, such tools might help them provide the support to their students they would like to.
Best,
Sarah
Ann Podleski
Ann Podleski
Totally agree - anything that can help teachers provide support to students is a plus!!! (And I don't know if this has the option for the teacher to look at what students are saying - again that would take time, but asynchronous, which is helpful. Also, the more I can see what students are thinking, the better I can help all students.
Sarah Bichler
Post-Doctoral Researcher
Hi Ann, thanks for keeping the conversation going! Yes, teachers can see the entire dialog students had with the avatar in their WISE teacher tools view. They can see the initial explanation, the adaptive feedback from the avatar and the student's response + how they revised their explanation. It does take time to go through the chats for all your students. I have seen a teacher look at some of them to get a better sense of how students are thinking about the science between class periods or at the end of the school day. It's kinda fun reading through. Other teachers looked at what specific students had said to see if these students needed guidance or not. Yet another teacher printed the chats and used them for discussion in the class! We're also thinking about using these chats during PD when teachers have time to really dive in and explore their students' ideas. But, as you're saying, it does take time!