NSF Awards: 1949122
Serious games have the potential to significantly enhance engagement, conceptual understanding, and procedural fluency in mathematics for all students, including students with learning disabilities and difficulties. Dream2B is a web-based video game designed with the Universal Design for Learning framework to maximize access and advancement of fraction knowledge for fourth through sixth grade students in supplemental mathematics instructional settings. The design of this asset-based mathematics intervention is shared.
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thank you for taking the time to watch our video! Our research team is currently testing our game-based ModelME curriculum with eight fourth and fifth grade teachers and their students. Because this research is ongoing, our video focuses on the design of the game (called "Dream2B") rather than specific research results at this time.
We are especially interested in discussion regarding the design and development of accessible game-based curricula that work from students' assets- that is, their diverse ways of thinking and making sense. We would love to hear if design features used in the Dream2B game might be applicable to your own context (e.g., work with students, teachers, and schools; research, etc.) and what those features might be. We also invite comments on any aspect of our work.
Allison Banzon
Lorna Quandt
Alejandra Duarte
Judy Storeygard
Jessica: Congratulations on this exciting asset-based work! The contexts seem engaging and provide rich and relevant learning opportunities. You made a wise decision to use the 3 minutes to immerse the viewer in the worlds instead of relying on descriptions from staff.
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thank you, Judy, for stopping by and for your comment! We are excited to share the game's design features and though the best way to do that was, as you said, immerse the viewer in the game worlds! Viewers can also play the game by visiting https://modelmemath.com/gamepage/.
Allison Banzon
Cassandra Hatfield
This is a really neat design of the games ~ connecting fractions to an interdisciplinary STEM context that goes beyond pizza and pies!
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks Cassandra! We fronted the mathematics in the challenges in terms of their focus on fractions, yet used the context of the careers to (hypothetically) increase student interest and engagement. We are excited to learn more in our current feasibility testing of the game based curriculum.
Allison Banzon
Corinne Brenner
Thanks for sharing this video! I'm curious if you're doing any data collection/analysis to see if students use different strategies to solve the problems. It seems like the visual/interactive assets could encourage trying out different strategies than paper worksheets, and tracking these differences during the design stage could be really interesting!
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks for watching and for your question, Corinne! I have two answers :).
First, we are collecting meta gameplay data in terms of looking at, for instance, different strategies players used, or if different number values or problem types across the worlds invite more diversity in strategy.
We are also looking at the data to better understand the assets and tools students use as they solve the problems (e.g., how different assets support strategies). We used Universal Design for Learning in our design of the user interface (e.g., tools, assets; sandbox gameplay features), so we are particularly interested in which aspects of the user interface enhance engagement, representation, action, and expression. I think this is linked to you idea of how the visual/interactive assets could engage students to take risks and try new strategies.
I'll also mention that a component of our curriculum (after game tasks) involves students directly unpacking strategies they used in the game in drawings and discussion, so we are excited to dig into those data as our study concludes this spring!
Allison Banzon
Lorna Quandt
Asst. Professor, Educational Neuroscience
Your video is really engaging and the game looks professional--great work! I love to see a foundational math skill like this packaged into something fun and engaging for students. You mentioned how you use UDL to guide the design and allow for a lot of customization of the game mechanics. I was curious about whether students organically use those customization features, or whether they tend to stick with the defaults. I also noticed that at some points, the screen is quite busy with content--words, images, and animations. Is it possible that some students will find the density of information to be overwhelming or make it harder to process and enjoy the game? Thanks again for showing this work--very exciting!
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks Lorna, for your questions and comments!
We've done two rounds of play tests with the game to date. In this work, we've learned that the customizable features students used quite often include the character customizer (e.g., changing how Bunny looks) and using the text to speech option for Bunny throughout gameplay. Other features were also used, yet the two I mentioned really seemed to stand out.
In regards to your question on the words, images, and animations on the screen, we did not see any initial evidence that this was problematic for players. Instead, early play tests pointed to the need for tutorials so that students understood what the sandbox game play tools were for and how they worked. Issues with screen busyness or density did not come forward in our interviews with students in terms of their ability to play or enjoy the game.
However, continuing to examine what features of our user interface are pathways or pitfalls to student engagement and learning is a great question that we continue to think about! We are in the midst of a feasibility study with eight teachers and their students (around 200 students). In this work, we are collecting meta level gameplay data alongside observational work of teachers' lessons and students' engagement in gameplay and lessons. We will be able to learn more upon analyzing these data and are happy to report any expansions of our initial play test findings at that point!
Allison Banzon
Myriam Steinback
Lorna Quandt
Matthew Marino
Professor
Hello Lorna,
Thank you for your comments! We are completing the preliminary trial of this game in the classroom with a replication study planned for next spring so these comments will likely change over time. We have a training tutorial for students where they demonstrate their ability to use the UDL tools prior to playing the game. In terms of the words overlaying the images, those were put in the video as a demonstration of the content being taught in the class. The students will discuss the vocabulary with their peers and the teacher. Playing the game is the best way to understand how it works. https://modelmemath.com/
Best,
Matt
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Great points, Matt! We do have a curriculum that wraps around the game, so to speak. Here, students preview aspects important for gameplay (e.g., tools; vocabulary), or a related STEM/ICT career for five minutes (usually through a quick video clip or application), then step into the game. After the game, students step into one of three after game tasks. Game replays give a space for students to generate their gameplay strategies and discuss them with peers. Worked examples give students a space to compare and analyze worked strategies through self explanation and conversation. Number strings allow students to worked on sets of problems with skills related to game play.
Sarah Carrier
Thanks for sharing this video. I appreciate your attention to students' deep learning and engagement in the stories of sustainable energy techs as they learn fractions by applying their work to authentic settings. Very nice!
Jessica Hunt
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks Sarah! We just got done with a few student focus groups this spring and have teacher groups next week; we're excited to dig into the data and learn more about how the narrative and careers connected to teaching and learning opportunities!
Allison Banzon
Kenneth Holman
Graduate Student
Sarah, thank you for your comments! Our main goal with Modelme was to ensure students' foundational understanding and comprehension of fractions while including STEM-based careers that students might not necessarily see or be aware of.
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Good point, Kenneth! Increasing interest in STEM and ICT careers is an important goal of the project.
Janet Johnson
This is a great description of the game! This makes it clear how diversity is included, STEM career information, and fundamental concepts for understanding fractions.
Allison Banzon
Jessica Hunt
Alejandra Duarte
Doctoral Student/Research Assistant
Thanks for your comment, Janet! I highly recommend playing the game too: https://modelmemath.com/gamepage
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks Janet!
Dan Roy
Research Scientist, Interest-based Learning Mentor, Learning Game Designer
Thanks for sharing. Looks like a lot of work has gone into this! A few questions:
How do you think about the learning curve. It sounds like based on usability testing, you went back and added a tutorial? How do you think about the density of concepts covered in each level?
How do you weigh introducing new ideas via text / spoken audio vs letting kids explore and experiment with feedback to discover?
How do you think about failure and feedback within the design, i.e. wrong answers from players? Those can be opportunities for surprise and delight, but does take a fair bit of time to implement well.
If you get more time to work on the game, what are some of the directions you'd like to take it in going forward?
Jessica Hunt
Allison Banzon
Alejandra Duarte
Doctoral Student/Research Assistant
Hi Dan,
Thanks for your questions!
1.) The tutorials have actually been in place from the beginning, but students' gameplay and their feedback in focus groups (within the last two days) have given us some ideas for how to make the tutorials more effective. I'm not sure I understand your question about the density of concepts, but the sequence of worlds is based on a fractions learning trajectory that Jessica Hunt developed and validated in a previous project (FAACT Learning Trajectory). Each world corresponds to a stage in the trajectory.
2.) I'll let others from the project weigh in here too, but from my point of view I have seen the feedback and discover aspect work well in the past for games. With Dream2B there are a few things to introduce or discover: the tools and their functionality, the objective in each world, and the actual fractions concepts. What I think are most important of those three for students to experiment with feedback and discover are the fractions concepts. Because of that, I think focusing students' cognitive load there and giving them visual/text/audio (UDL- multiple means of engagement & access) scaffolds for figuring out the tools and the overall objective of the task is appropriate.
3.) From a mathematics standpoint, we are very much seeking to leverage wrong answers from players - as you mention this is something we are trying to do within gameplay, but this takes a lot of time to implement from a game design standpoint. Some other ways we accomplish this are based in the fact that the game is embedded into a larger curriculum, ModelME. The ModelME curriculum includes question prompts for teachers to respond to students' strategies during gameplay. Also, students have the opportunity to discuss "partial" or "alternative" conceptions for a task to further make sense of problems they are encountering in gameplay.
4.) There are lot of things for the game moving forward, but one that we didn't have available in the Beta version but is in the works is a teacher dashboard. We welcome any thoughts about key elements to include or make prominent on the dashboard to support teachers' instructional decisions!
-Alex
Jessica Hunt
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Hi Dan!
Thanks for your questions!
As Alex mentioned, we think about sequencing/density of concepts in each world as a stage of a learning trajectory on which this project is based (Hunt et al., 2020). Learning trajectories are distinct in that they are not so much a task analysis but more a system, if you will, that contain a goal(s), developmental stages of thinking, and activities designed to promote the stages of thinking. So, within each world, students experience a series of carefully sequenced tasks with varying structures, number values, and representations designed to support and extend their reasoning.
I'd say that our current version of the game is based on encouraging kids to be active (solve problems by using game actions and tools) with cognitive feedback to support them to notice and reflect upon their game-based actions. We do include many opportunities in the curriculum (which contains the game but also contained after game tasks) for students to bring forward strategies used in the game, self-explain, discuss with a partner, make revisions, and share as a group. Within the game, we use UDL principles to support students to access, build, and internalize their reasoning.
Moving forward, we are working over the summer to analyze the data from the feasibility study, including students' and teachers' focus group data, meta data on gameplay, teachers' integrity when using the program, and written pre and post measures of fraction concepts, engagement, and STEM Interest. We plan to synthesize these data to inform our final build/iteration of the game. Stay tuned!
Best
Jessica
Allison Banzon
Marcelo Worsley
Assistant Professor
Thanks for sharing this neat project. As you mentioned doing user testing, what are the kinds of in-game and out of game measures that you are looking at. I am thinking that many affective measures could be useful. I also wonder about something like Valerie Shute's stealth assessment as a way to measure learning during the game play.
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Hi Marcelo!
Thanks for your questions! I will defer to Michelle, Allison, and Matt for the in-game measures, especially as they may relate to stealth assessment and affect (I know Michelle has much to say on this!).
For out-of-game-measures, we are using the Fraction Schemes Test (Wilkins et al, 2013) as a proximal measure of the program's effects on students' fraction concepts. It measures fraction concepts along the learning trajectory on which the game is based.
For engagement as students take part in the ModelME curriculum, we are using the Engagement in Science Learning Activities Survey (Chung et al., 2016). It is used to measure a student’s cognitive, behavioral, and affective engagement. Participants respond on a Likert-type scale ranging from 1 (YES!) to 4 (NO!). “Because no particular assumptions are made about a task structure other than there is a particular task that should have been completed (p. 1)”, valid inferences can be made regarding overall mathematics engagement after activity using responses across all of the items.
To measure STEM interest, we are using the Upper Elementary School (4-5) Student Attitudes Toward STEM (S-STEM) (Friday Institute for Educational Innovation, 2012) supplemented by semi-structured focus group interviews with a subsample of 40 students.
Best
Jessica
Ho-Chieh Lin
Allison Banzon
Allison Banzon
Graduate Researcher
Hi Marcelo, thanks for these questions!
Right now we're working to develop a gameplay data pipeline that will work in the background of Dream2B to generate in-game, process oriented measures to compliment the existing out-of-game measures Jessica mentioned above. The pipeline will produce data on students' clickstream behaviors within the game. We can then use this data to analyze in-game behaviors (e.g., tool use frequencies within levels), identify common game play paths and associated learning outcomes, or interpret instances of self-regulated learning behaviors during gameplay.
While we don't have plans to interpret this data through an affective lens just yet, we are working on populating this process-level data to teacher facing data dashboards. We'll collect initial feedback on what kinds of student data and data visualizations teachers would like to see included in these dashboards during our upcoming teacher focus groups!
Jessica Hunt
Hiller Spires
Hi Jessica and team - Dream2B is an innovative, engaging context for students to understand fractions. Congratulations on your design--I know a lot of effort has gone in to this! Good luck with data collection,
Jessica Hunt
Allison Banzon
Allison Banzon
Graduate Researcher
Thank you, Hiller!
We're looking forward to reporting on our initial findings as play tests wrap up in the coming weeks. We're excited to examine how Dream2B impacted students' fraction comprehension while connecting them with STEM careers they may not have otherwise been exposed to.
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks Hiller! We are excited to analyze the data!
Allison Banzon
Ho-Chieh Lin
Hi, I am Ho-Chieh Lin, a Ph.D. candidate at OSU. My work involves "videos" and fractions, so I am very excited to learn about your team's project. I believe the video games you have developed are attractive to children. I was wondering how you suggest teachers implement these activities in their classrooms. Do they use them for exploration, assessment, or review activities? What if children could not figure out the chanllenges? Also, I was curious about how you decided on the topics of each task. I love how the contexts connect with STEM fields. Look forward to your reply :)
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Hi Ho-Chieh, Thanks for stopping by and for your questions. Our game is one of three parts of a supplemental curriculum for fractions. As Alex mentioned, there are three parts - the preview, the game itself, and the after game tasks. The preview is meant to unpack critical ideas students will see in gameplay in a particular world, or to tie to the STEM/ICT career of that world. The gameplay is sandbox, puzzle play where students solve problems and gain feedback on their strategies from the game and from their peers and teacher. The game supplies cognitive feedback along the way and also provides optional hints students can use as the play the game. The tasks are set along a trajectory developed during my previous CAREER work at NSF. The after game tasks give students opportunities to discuss their gameplay strategies and extend their reasoning.
Alejandra Duarte
Doctoral Student/Research Assistant
Hi Ho-Chieh - great questions!
As far as teachers implementing the activities, the Dream2B video game is embedded within a supplemental fractions curriculum, and we see the "launch" and "after gameplay tasks" as integral to students' gameplay. We've suggested 10 minutes of gameplay at least 3 times per week in our feasibility study. The set up of the game is a type of "sand box" play so in that sense students can use the game as a form of exploration. Teachers can also use students' gameplay as a form of formative assessment by observing and questioning students as they play the game. Soon we will have a teacher dashboard developed that also gives teachers data that can support formative assessment.
When students struggle with the challenges, there are questioning prompts teachers can use to learn more about what they've tried so far and support them thinking about other ways to try to solve the problems. The game also gives text prompts to students as they play depending on the actions they take. Perhaps most important here is the follow up that happens in the after gameplay tasks, which are essentially lessons rooted in a number string, worked example, or "game replay." During this time, students have a chance to think, write, and discuss real problems from the game and teachers facilitate those conversations. Our hope with this is that these conversations support students with difficulties they had in the game the next time they play.
I'll let Jessica share more about how the task topics were decided. Thanks for your compliments on the game!
- Alex
Allison Banzon
Patti Parson
I appreciated seeing the actual game design (mom of now-grownup gamer). My work isn't in the same field as you (ie more adult informal science ed), but it was enjoyable seeing what you were doing.
Allison Banzon
Jessica Hunt
Associate Professor, Mathematics Education and Special Education
Thanks, Patti! I appreciate it. We're excited to dig into our data this summer toward the next iteration and test of the game. -Jessica
Allison Banzon
Graduate Researcher
Thank you, Patti! We're excited to begin analyzing the data!