NSF Awards: 2101441
This DRK-12 Impact Study project will investigate the effectiveness of STEM-Innovation and Design (STEM-ID) curricula in approximately 29 middle schools, targeting 29 engineering teachers and approximately 5,000 students across middle grades in Georgia. This 3-year middle school Engineering and Technology course sequence was developed and implemented as part of the AMP-IT-UP (#1238089) Math/Science Partnership. The project-based learning curricula align with national math and science standards, introduce students to advanced manufacturing tools such as computer aided design (CAD) and 3D printing, incorporate engineering concepts such as pneumatics, robotics and aeronautics, increase student awareness of career paths, and address the concerns of technical employers wanting workers with problem solving, teamwork, and communication skills. Our published research, conducted in collaboration with a low-income, rural fringe school system, demonstrates unequivocally that enabling students to practice their science and math skills and knowledge within the context of interesting and engaging middle school engineering classes significantly benefits both academic achievement and engagement in STEM. The research also demonstrates that across grade levels and schools, students are able to transfer knowledge between engineering courses and their core mathematics and science courses. This impact research study will determine whether STEM-ID courses are equally effective across different demographic groups and school environments under normal implementation conditions and whether the courses have the potential to positively impact a vast number of students around the country, particularly students who have struggled to stay engaged with their STEM education.
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Welcome to the STEM-ID showcase!
I am a research faculty at Georgia Tech and PI of the NSF DRK-12 STEM-ID project. Our video highlights the middle school engineering courses (STEM-Innovation and Design (STEM-ID). STEM-ID aligns with national math and science standards, promote the use of the engineering design process, introduce students to advanced manufacturing tools, incorporate engineering concepts, increase student awareness of career paths, problem solving, teamwork and communication skills. Three year middle school course sequence was developed as part of another NSF funded project, AMP-IT-UP. In this new DRK-12 Impact study, we plan to implement the curriculum in 29 middle schools to study the impact in a larger setting.
Please check out the curriculum site, STEM-ID, you can request the full curriculum by sending us an email at ampitup@gatech.edu.
We look forward to your comments!
Meltem
Gerald Knezek
This project serves a very important need. I would love to know more about your measures (best email: email gknezek@gmail.com).
Your philosophy fits very nicely with our colleagues at the STEM Pre-Academy, Univ. of Hawaii where they serve 700 middle school teachers. I have sent your summary paragraph to Cheryl Ishii the Director.
Nice project!
Gerald Knezek
simEquity
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you, Gerald. Please check out our publications from AMPIT-UP, you will have a better understanding of the student measures. Since the new study focuses on replicating the original study, we plan to use the same measures. You can also request the curriculum, we are always looking into more implementation sites! I will also send you and email, so we can connect!
Adem Ekmekci
Engineering design courses are of a great need at middle school level. It's great to see this project. It seems like this would improve students' 21 century skills, too.
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you, Adem. We are excited about the opportunity to scale up our project, and capture the impact.
Amy Wilson-Lopez
Associate Professor
Thanks so much for this informative video, and congratulations on your recent award that scales your prior AMP IT UP project! When watching this video and reading this abstract, I had two questions that came to mind. The first was surrounding the phrase "interesting and engaging." How do you select or create engineering design challenges that are interesting and engaging to youth? This question becomes even more important when you are seeking to broaden participation among youth with different cultural and linguistic backgrounds. Second, are the engineering courses mandatory or optional for youth? If they are optional, what strategies do you have in place for recruiting students who might not historically have enrolled in these courses? Thanks again for sharing this work, and I look forward to reading the findings from this study!
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you, Amy. Great questions! AMP-IT-UP project was 7 years long, which gave us enough time to iterate and change the curriculum and figure out some interesting and engaging design challenges. Through the implementation data, so far we learn that they are still engaging for the youth. I agree that with with the changing world, this could be important to look at. The original study was in 4 middle schools in one district, with the scale up, we will be in 29 schools, hopefully we will know more about this. Part of the grant focuses on adapting the curriculum to new settings, which will allow us to make some revisions.
In GA, there are connection classes, one of them is Engineering and Technology. In some schools, students are put in this class randomly, in some schools it is required for all students. In our original study, we had all levels of students in the study, which helped us in our research design and coming up with a comparison groups.
Thank you for visiting our site! We appreciate the thoughtful comments and questions.
Marion Usselman
Associate Director, and Principal Research Scientist
Amy--As Meltem mentioned, we spent several years working up scenarios and story lines that would maintain student interest for a full semester. The original AMP-IT-UP grant had an advanced manufacturing focus and was targeted towards students in a low income, rural fringe school system. Each course (6th, 7th and 8th grade) requires its own set of engineering equipment that helps facilitate students learning a specific set of engineering skills and exposes them to technologies common in manufacturing. During each course students also engage in an open-ended design challenge that requires that they fully grapple with foundational math and science skills and with the engineering design process. The equipment, which includes 3D printers, LEGO robots, and pneumatic catapults, is a substantial investment by the schools. Though it constrains the types of scenarios that can underpin the design challenge, we anticipate that the specific story lines will change with different audiences. What is important is that the design challenge requires students to engage with the math and science to be able to come up with an answer, and that different student groups can come up with divergent designs.
Thanks for your interest. We are happy to share the curriculum for anyone interested.
Julia Varnedoe
Amy Wilson-Lopez
Associate Professor
Thanks for this response! Just brainstorming here along with you. If some students have not flown a plane, they might be motivated by scenarios in which a plane was used to achieve an important or relevant purpose. I know drone technologies have really taken off in the last several years too--we are working on a team with engineers who use drones in different applications--one to look at damage to individual plants in crops, one to survey dams to see if there are any cracks, and one to look for an invasive plant species that is contributing to wildfires in Utah (it grows very quickly and then dries out). Designing a foil for a drone (in addition to or instead of an airplane) might be a good way to do something similar, while then opening the door to a huge range of applications that could be applied within students' own contexts and communities, so they can have an immediate sense of how engineering makes a difference to them. Again just totally brainstorming here--I don't know if something like this could work for your project or not. Thanks again for sharing and best wishes as you move forward.
Scott Pattison
Research Scientist
Great video. Thanks for sharing the project. The video talks about ensuring that the curriculum works for a diverse range of students. I'm curious to hear more about the strategies that are built into the activities and teacher resources to support this goal. I'm also curious if you've looked at identity negotiation in the classroom around these engineering activities. I did some work on identity and engineering in afterschool settings, so I'd be interested to hear any reflections you have on this topic. Thank you!
Marion Usselman
Associate Director, and Principal Research Scientist
Thanks for the question, Scott. We developed an extensive amount of curriculum for the AMP-IT-UP MSP project. This included the STEM-ID engineering courses as well as 1-week modules for use in the middle school math and science classes. (For an example of the modules, see our NSF video at https://multiplex.videohall.com/presentations/1... All of the curricula were designed using PBL practices and pedagogy drawn from the science education literature and from How People Learn. Our curriculum design team was rooted in projects such as Learning By Design (Kolodner et al), and Project-Based Inquiry Science. All activities promote constructivist, collaborative learning and require students to participate in extended, guided inquiry and design challenges that lead to non-convergent solutions. Within the curriculum, teachers are able to arrange groupings and adjust the level of scaffolding to support a diverse range of learners. This worked well within the initial four middle schools, which were all challenging school settings, and with the handful of additional schools that have adopted the curriculum. One of the goals of the current project is to identify whether different issues arise in a completely new setting.
We did not look directly at identity negotiations in these classes, though we did build in different roles that rotate between students within their groups. Meltem led up the research in AMP-IT-UP, so might have more to say about this.
Scott Pattison
Research Scientist
Thanks for this additional information, Marion. All very interesting. I look forward to hearing more as you move forward on implementing the project in new settings.
Nidaa Makki
Professor
Thank you for sharing this project! This seems like a very interesting program that engages students in various aspects of engineering design, including the use of technology. Scaling up the implementation of the curriculum to 29 schools is wonderful opportunity to collect data on how it’s working in various settings. I appreciated that you incorporated a case study approach in your research design.
I am curious about how you are measuring academic engagement. Do you have specific instruments you are using for this purpose? Are you developing an observation protocol for the case studies?
You also mentioned in the video that you are measuring the impact of the professional development. Can you elaborate more on the PD that the teachers receive? It would also be interesting to hear a little bit more about the feedback from teachers who previously implemented AMP-IT-UP courses. Were there any specific challenges they identified?
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you, Nidaa. Great questions! We have a validated student engagement instrument that measures 3 dimensions, cognitive, behavioral, and emotional engagement. Through AMP-IT-UP project, we were able to validate this instruments, where we found significant results. Check out our publication, we explain it further.
For the teacher PD, we are providing 4 days summer professional development (in person), later in the year, we will support teachers through online collaboration. In our original study, we had 4 teachers and worked very closely with them over 5 years, so part of the new grant, we want to develop a more robust PD, including more PD materials. One of the challenge that teachers mention when they first implement it, it is hard to finish the curriculum, they run out of time at the end of the semester. Another challenge often mentioned is navigating the group work, but mostly teachers were able to adapt to curriculum to their teaching.
Nidaa Makki
Professor
Thank you for sharing the article! I like that you measured engagement across the cognitive, behavioral, and emotional dimensions.
We also found in our work that in person PD was valuable to engage teachers with the hands-on components of engineering design. While online PD can be scaled more easily, it requires more time investment on the teacher's end for troubleshooting. Very interesting work, good luck with the scale up!
Meltem Alemdar
Karen Hammerness
What an exciting project, and so impressive that you've had this ability to implement the curriculum and program in multiple schools across the state. We also work with middle school and high school students, although we have more of a focus upon science practices vs. engineering. I noticed that as one outcome measure, you use standardized test scores in science and math--we also use those measures in a number of our programs at the museum, but we often struggle to help a larger audience understand the effect size and what any kind of statistical impact means...especially if it seems relatively 'small.' I'm curious about the effect sizes you've seen and how you have supported conversations about how to make sense of the reliability and validity of test scores as an outcome, as well as to help people not view it as the 'only' important outcome.
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you, Karen. yes, effect sizes are hard to explain, when we talked about the results to our stakeholders, we focused on explaining group differences. Students in our original study were put in the engineering class randomly, so we were able create comparison groups. Here is the paper that explains it further. With the larger sample size in our newly funded project, we are hoping to do more advanced modeling.
Lizzy Cowan
Very interesting work and great video! Is the graph on 00:28 of your video measuring STEM engagement? Why do you think Year 2 and Year 3 had no change for Math? I just requested a copy of the curriculum. I'm very interested in learning more!
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you for visiting our site. We found that participating 2 years or more makes a significant impact on the test scores. we some more impact on the science side in terms of test scores, but the cut point was taking the courses at least 2 years during the middle school. Here is the published study about it.
Thank you for requesting the curriculum, please let us know if you have any questions. We are always looking for sites to implement the curriculum.
Jackelyn Lopez Roshwalb
Thank you for sharing your meaningful work! I am curious about your 29 engineering teachers. Do they regularly teach only engineering, or are they primarily math and science teachers taking on these new courses? I also wonder, how do schools "fit" these courses into an already filled day? Are they full on courses, small projects that are embedded in their science and math courses, or something else entirely?
Meltem Alemdar
Principal Research Scientist/Principal Investigator
Thank you for visiting our site. In GA, we have connection class in middle schools for Engineering and Technology. The curriculum implemented in those classes, and all are Engineering teachers. The curriculum aligns with the engineering standards. The Engineering teachers fully implement it in their classrooms.