NSF Awards: 2100784
This study addresses two open questions in mathematics education and teacher learning research related to groupwork monitoring. Using contemporary information visualization techniques and open-source tools, alongside a video-based coaching activity, teachers will a) analyze classroom video records featuring group math discussions and b) uncover and investigate their specific interactions with student groups as well as their overall approach to this important phase of their lessons. Through these tools, teachers will develop strategic and integrated understandings of effective groupwork monitoring strategies. As a result of this work, teachers and researchers will be able to better connect teachers’ monitoring choices to students’ peer-to-peer math talk.
Nickolay Hristov
Senior Scientist, Associate Professor, Director
Hello TAU Team! I was intrigued by your use of visualizations to inform the work of this project. The last segment of your video was informative how this might happen. Is there evidence that movement pattern in the classroom is an important variable to measure, automate, and in this case, visualize?
Kristin Flaming
Sue Allen
Senior Research Scientist
Yes I wonder this too - it's innovative and intriguing but if the teachers also have video of their interactions with the students, what do you see as the value-added of this representation?
Elizabeth Metts
PhD Student
This is a great point! We are still learning, but when we've used the representation with teachers, the IGS serves as an aggregation of the teachers' movement, and supports them to think more abstractly about their practice. In the interviews with teachers we have done so far, they have found it super meaningful and generative -- so we are very excited to continue to explore more.
Kristin Flaming
Ilana Horn
Professor
teachers are able to see how they distributed their attention across groups. they can see the varying lengths of interactions with particular groups. they can use the representation to say, "i wonder what that long interaction was about? why did i spend so much time talking to them?" and then click over and see the video of the interaction. it gives them a map for navigating the overall lesson.
Kristin Flaming
Sue Allen
Senior Research Scientist
Got it, thanks! It sounds as though it could be a helpful tool from an equity perspective - is the teacher giving more attention to certain groups without realizing it. Though I suppose it would take work to keep track of which students were in which groups at which times. Have you run into TeachFx? They've automated some of this in their discourse analysis tool - worth knowing about, though we found it wasn't quite matched to our own needs...
Ben Rydal Shapiro
Assistant Professor
Hi @Nickolay and @Sue great to meet and thanks for all your thoughts and posts. If of interest, to add to this thread, it's worth saying that the representations/tools we are developing with teachers are highly interactive so they support a lot of different types of exploratory data analysis as opposed to being static representations. Also relevant is that we continue to develop specific types of interactive and visualization techniques that teachers and our team feel are valuable/benefit their work or teachers' reflective practice. @Sue some of the origins of the tools we're developing were in museums so it would be really neat to stay in touch with you particularly with all the amazing work you've done in museums!
Channa Comer
STEM Educator
Hello and thank you for sharing your work. Your project appears to combine research approaches that advise circulating and listening with those that advise getting involved through the use of the movement graph and the videos. You can see how teachers circulate and hear how they interact with students. I think the information would be very valuable for a teacher to get feedback on whether they are equitable in their circulation and the quality of their interaction with students as they move around the room. This is in addition to highlighting and identifying dilemmas that may come up during group work. Can you describe the process of creating the movement graph including the technology that is involved?
Elizabeth Metts
PhD Student
We use an open-source, web-based software called Mondrian Transcription: https://www.benrydal.com/software/mondrian-tran... developed by one of our PI's, Dr. Ben Rydal Shapiro. We typically upload a video to the software to transcribe (which takes our team about 15-30 minutes, depending on the length of the class period), but we have also been experimenting with live transcriptions too.
Ben Rydal Shapiro
Assistant Professor
Hi @Channa, thanks so much for your post! We've developed the visualization tool to work with either automated/sensor-based technologies as well as manual transcription techniques that Lizi shared above. The advantage of the manual techniques is they provide very precise location data in classrooms as sensor-based technologies remain very imprecise in classroom settings.
Sue Allen
Senior Research Scientist
I'm intrigued by the phrase "effective monitoring" in this context, particularly given your fascinating point that the research has contradictory recommendations. How will your teachers know what is effective? Is student learning being assessed in some way as part of the project? Thanks!
Elizabeth Metts
PhD Student
Our conceptions of "effective" and "good teaching" are driven by the teacher's own understandings of good teaching and their sense of teacher identity. Not to be relativistic, but we aim to center their own understandings and goals so as not to be evaluative.
Ilana Horn
Professor
@sue part of the richness of our video coaching intervention is that teachers can listen to student talk before and after their interactions with the groups. this allows them to assess for themselves how their interactions supported students' math discourse or perhaps made it less rich. our pilots have been really great and eye-opening for the teachers
Sue Allen
Senior Research Scientist
Great point - I'd love to hear more about what kinds of things the teachers are inferring from these videos - and do they draw similar or different conclusions about effective practice?
Hui Yang
What a great presentation! I am looking forward to learning more about the findings once they become available. In the meantime, I am curious to know your thoughts on to what degree the different math tasks that students are working on might influence teachers’ decisions on providing the kinds of guidance/feedback on students’ collaboration work. Thank you!
Ilana Horn
Professor
that is one of the issues we are looking into. we suspect that, as previous research suggests, some tasks are more "discussable" than others. we have also noticed that it seems to differ at different points of the lesson... at the end of lessons, teachers are more likely to do things to move students along.
Sue Allen
Senior Research Scientist
I do have one more question, about learning outcomes. I realize this is a very new project, but could you share what your plans are for assessment of what the teachers are learning? Thanks.
Ilana Horn
Professor
in our previous study (Project SIGMa), we assessed teachers' learning by following their thinking about practice and their instruction. it was a very case-specific assessment, and we found that 11 of our 12 focal teachers showed evidence of changes in instruction and conceptualization of their work on a one year follow up. we wrote about our methods and findings here: https://www.routledge.com/Teacher-Learning-of-A...
Kristin Flaming
Are you gathering data from the students to see how their perceptions of their teacher and own learning changes as the teacher integrates what they learn from their movements?
I would be curious if as the teacher more evenly distributes their time across groups if the struggling students/groups feel they are not getting enough help and if those that are doing well feel like the teacher check-ins in too much. I teach a project-based approach to introductory statistics, Passion-Driven Statistics, that is a flipped classroom. I am constantly moving around the room as students as for help and then checking on those that have not ask questions. I find that the students that are doing well quickly explain how it is going and are like I am good you can help someone else. Then the students that are struggling like me to sit down and walk through some of it with them. I feel like if I more evenly distributed my time I would distract and annoy the higher performers and my struggling students would not get caught up.
Ilana Horn
Professor
that is a really great question, kristin. we aren't interviewing students for two reasons. first, there is a team capacity issue. although we have several research assistants and a postdoc working on the data collection, we don't have the bandwidth to do justice to these questions. second, because the students are minors, they are a protected population. we have found that they are willing to be filmed while they are in class, but the IRB requirements to get them to assent to interviews (as well as their parents' consent) would be really challenging in this district. the district is understandably concerned about lost instructional time, so pulling them out of class would not work. additionally, transportation issues are complicated for a lot of our students, so asking them to come before or after school would be difficult as well. it would make for a great follow up study though!
Kristin Flaming
Kristin Flaming
I completely understand as we work with high school students sometimes with our project. I do think any change done in the classroom should be for student or instructor benefit. Based on your video it would seem that the teachers being aware of their movements and making change to improve those is more of an impact to the students than the teachers. Since your focus is on the teachers, what benefit do you see for the teachers and how will you know if it is effective for the students? Will you review grades or class distributions on projects to see they improve as the teacher makes changes?
Ilana Horn
Professor
we have 360˚ cameras on a subset of student tables. we are examining the student talk prior to and following teacher interventions, with an eye on both the academic and social dynamics. we hope that we can use those data to ascertain the quality of their engagement and how teachers' interactions with the groups enhance or diminish that engagement. does that help?
Kristin Flaming
Sue Allen
Senior Research Scientist
Interesting discussion. I agree that the student discourse data is likely to be very revealing, even without additional non-embedded assessments like interviews or surveys. And presumably it includes comments from the students directly to the teacher of the kind Kristin is talking about, so that would be grist for the teacher discussion afterwards, and also the analysis.
My last thought is related to Ilana's last post... Given that you're doing analysis to try to make causal arguments between teacher interventions and QUALITY of student talk, it does seem as though you are being evaluative, just not sharing that with the teachers. Is that right? For me it brings up an interesting methodological question for these kinds of iterative reflective interventions: do you say nothing to the teachers and let them draw their own conclusions while you may come up with quite different conclusions after doing deep analyses? Or do you include your team's analysis as part of the intervention, providing some kind of rough cut of what you're seeing to inform the teachers' discussions? (I think Catherine Haden & Maureen Callanan call it 'blitz coding' so that it's quick enough to be formative feedback.) Just a thought... I look forward to hearing more about the project as it unfolds.
Ilana Horn
Professor
Sue, I so appreciate your engagement with our project! we are using these representations (the video and the IGS) to do co-inquiry with our teachers. part of what we deliberately avoid is evaluative talk. we talk about it a lot in chapter 6 of our book, "Learning to Inquire into Teaching". feel free to reach out offline too :)
Kathryn Kozak
This is an interesting study. Have you thought of expanding this to higher education. I know that many community colleges are using active learning techniques in their classes to help their students understand the material. Also, my project, StatPREP, has active learning activities for teachers to use in their classes.
Ilana Horn
Ilana Horn
Professor
thank you kathryn! i am sure it would be interesting to use this approach in community colleges as well