NSF Awards: 1620979
Given the costs of buying an automobile, consumers often opt to get regular tune-ups to keep the car in good condition. The cost of tune-ups tends to be low compared to the initial price combined with potential costs of major repairs. The value of automobile tune-ups is well-established and linked to the importance of long-term function. In contrast, professional development programs often invest significant resources in high-quality initial programs but fail to find ways to extend the positive outcomes over the long term.
To determine the value of a professional development tune-up, we provided modest, follow-up supports to 49 teachers who had previously completed science-based professional development. Several years after their initial professional development, we found declines in teachers’ sense of preparedness to teach science, their self-efficacy, their instructional time in science, and their use of inquiry-based science instruction.
Supports included two-day “refresher sessions” related to implementing NGSS; electronic supports; face-to-face meetings; and $150 per year for science supplies. These modest supports stopped the declines in teachers’ sense of preparedness to teach science, their self-efficacy, their instructional time devoted to teaching science, their use of targeted instructional strategies, and their implementation of inquiry-based student activities in science. The supports not only stopped these declines but reversed them. By the end of three years, the outcomes increased almost to the high level achieved at the end of the prior intensive professional development programs. Our research suggests that professional development tune-ups may offer a cost-effective way to sustain instructional changes over time.
Cathy Ringstaff
Senior Research Associate
Hi everyone. Thanks for dropping by to watch our video. We hope it sheds light on the importance of offering support to teachers after professional development ends.
Let us know your thoughts. What other supports could be offered to teachers to help sustain professional development outcomes? What supports would teachers find most effective? What supports would be easy to scale across different districts?
Nickolay Hristov
Senior Scientist, Associate Professor, Director
Hi Cathy and Team! As someone who maintains a vehicle and designs professional development programs in informal settings, I relate to the challenges of keeping the benefits of the initial effort over time. Did you experiment with the optimal length of time since the end of the original training for the “tune up” intervention? If you started sooner than 1 year, was there evidence that it might not take as long as 2-3 additional years to reach those original outcomes (if I understood your data correctly at 02:20-02:30). In other words, is it possible to avoid that slump at the end of the first year altogether with something closer to the original professional development. Your findings make a lot of sense and should be a common sense in sustained program support, however, it remains difficult to ensure continued funding after the end of a project.
Cathy Ringstaff
Senior Research Associate
Hi Nickolay! Thanks so much for watching our video and posting your question. You have interpreted our findings correctly. We were not able to do an experiment to look at the optimal length of time to wait before adding our modest supports. The four original PD programs that were offered ended at various times, and were funded by the State of California. I was the evaluator on all of them, and used the same measures on all of them. When the projects ended, I was curious about whether the changes made were actually sustained. So, Dr. Sandholtz and I did an NSF study to look at the long-term impacts on teachers from one of the groups (https://www.nsf.gov/awardsearch/showAward?AWD_I...). When we saw the PD outcomes declining in this study, we began to wonder if there were modest supports that could be offered to teachers to stop the decline.
I personally believe that this slump could be avoided. In fact, I am working now on a study with new teachers in rural North Dakota, Wyoming, Montana, and California that will offer PD and modest supports to see if we can learn more about this topic. Although it seems pretty obvious that this kind of support would help teachers, we could not find any existing research about this. The funding was just awarded. If you email me a reminder in a year, I'll let you know what we are finding!
Sue Allen
Senior Research Scientist
Yes I agree - the timing question would be great to study in this next project if you can, because a project might only have enough resources for, say, one tune-up for a cohort of teachers, and it would be super helpful to have some idea of when to do it. I don't think it's necessarily true that it should even be at the beginning of the slump - maybe you wait a few years while their retention is still pretty good, and only tune up when they're really starting to forget a lot... it sounds a bit unintuitive but maybe you get sharper improvements if you tune up when they've forgotten more. I guess you're trying to maximize the area under that effectiveness graph (as a measure of overall quality of teaching over a career, say).
Cathy Ringstaff
Senior Research Associate
Hi Sue. Thanks for the feedback on the project. You make a great point about timing. I will talk to the other PIs about this issue and see what they think. Perhaps we could do a bit of an experiment and vary the timing between teachers in different states or between other groups and see what we find out. So much of the support issue boils down to funding.
Rebecca Sansom
Thank you for sharing your work! I have a question about the follow up and supports. In our PD program, we have a summer workshop followed by four follow up meetings in person and extensive virtual interactions with a team of teachers over the course of a year. We haven't built in ongoing support after the year of participation, however. What would you recommend based on your research?
Judith Sandholtz
Professor
Rebecca, Our research suggests that continuing some ongoing follow-up support (for as long as possible) will help in sustaining outcomes from the PD. If you don't have funds to add follow-up support for subsequent years, you may want to consider shifting some of the planned follow-up meetings and virtual options from the first year to later years. Some follow-up supports can be maintained at a relatively low cost.
Cathy Ringstaff
Senior Research Associate
Hi Rebecca! Thanks so much for your great question.
In our first NSF study about the sustainability of teacher development (#1119589, https://www.nsf.gov/awardsearch/showAward?AWD_I...), we found that teachers’ attitudes and instructional practices began to decline two years after the professional development ended, but remained higher than pre-program. The most significant declines occurred in teachers’ self-efficacy in teaching science. Teachers continued to use a broader range of instructional strategies in science than pre-program, but their reported frequency declined. Teachers’ most frequently requested forms of support were modest, a finding that led to the study described in the video.
I would actually recommend starting modest supports toward the end of the year of their professional development, so the teachers have the opportunity to experience these supports and get in the habit of using them. For example, one of the supports teachers used most frequently was our monthly newsletter, full of resources about teaching NGSS. We also held webinars were they could chat and collaborate, but these were not used by many teachers. (I believe these webinars would be more popular now that teachers are used to Zoom; when we did them, it was before the pandemic, and Zoom was not popular.) I feel like if you start these types of supports before their participation "ends" at the end of year one, and they are continued as the project continues with new teachers, that will help sustain the teacher outcomes.
We will happily share our newsletters with you to adapt if that would be helpful. Creating relevant newsletters can take a lot of time, and this might help you.
Rebecca Sansom
Cathy, Thank you so much. That would be really helpful to see what the newsletters contained and what topics you used for the webinars. Thank you for being willing to share!
Cathy Ringstaff
Senior Research Associate
Hi Rebecca. I will send some to you via email. I think you will be able to use some of the resources we found for your teachers!
Channa Comer
STEM Educator
Hello and thank you for sharing your work and providing another data point to support the efficacy of ongoing professional learning rather than "one and done" models. Your study focused on elementary teachers. What might you expect to find if you repeated your study with middle school and high school teachers? Have you explored the possibility of grooming teacher facilitators as a means to help teachers retain their self-efficacy and skills? I have always found that having to teach something is a great way to gain mastery.
Cathy Ringstaff
Senior Research Associate
Hi Channa. Thanks so much for watching our video. I would love to do more research on middle school and high school. We focused on elementary grades because the PD that the teachers participated in before our study (and from whom we had existing data) were primarily elementary teachers. I suspect that all teachers could benefit from the types of support we offered. We stuck to offering the support to elementary teachers in this study because we find that elementary teaching (where teachers usually don't have to teach science) need different types of support than middle or high school science teachers.
In the original studies of the PD (on which I was the evaluator), the projects did have teacher leaders to help teachers in their districts. Some of them went on to become Science Specialists within their districts, while others did very little to help their colleagues. We did not have teacher leaders in this current study, but I totally agree that having to teach someone something is one of the best ways of learning.
Sue Allen
Senior Research Scientist
Sounds like a terrific project, and I love the fact that it's applicable across all kinds of specific PL programs. I also think it's fascinating that some of the electronic offerings were more used than others - maybe some teacher interview data on that might be a super helpful addition to understanding what works and why, among the set of offerings you provided. Could it even be the $150 for essential supplies that has the most impact? - e.g. Do they know that comes from the program that runs the PL they took? (ie. might it be an affect-loaded reminder of the program and its principles?)
Cathy Ringstaff
Senior Research Associate
Thanks, Sue. I think it is very interesting that many of the teachers in this study were very resistant to using webinars or Zoom. We had to train them repeatedly on how to use these supports. Now, after the pandemic, I suspect our findings would be much different!
We do have a lot of interview data that we didn't have time to discuss in our video that describes some reasons why teachers didn't avail themselves of some of the supports we offered. As you probably would suspect, teachers often felt they did not have time to participate in webinars and Zoom collaboration. Many seemed intimidated by the technology.
They seemed to find the newsletters valuable because they could read them at their convenience.
The teachers did know that the money for supplies came from the project. However, some of them were reluctant to order anything, for fear that it would take a long time to get reimbursed by the project. We ended up ordering for these folks, once they told us what they wanted. We certainly had not anticipated this as a barrier.
Yes, these lessons ARE applicable across all kinds of PL. Interestingly, many people say to me, "Well, of course support is needed after PL." If is so obvious to people, why is it rarely offered? And why are there so few longitudinal studies about outcomes of PL once the PL ends?
Sue Allen
Senior Research Scientist
Interesting. I suspect the short-term funding cycles are a big part of the answer to both questions. But the power of this approach is that it's such a light touch, it might be fundable through normal operating expenses after a grant ends. Do you have any sense of how much time it takes the project team to create one multi-modal "tune up" using the approaches you tried? (labor costs are usually the killer).
I wonder if there's a way to do some kind of think-aloud protocols (or maybe click-streams) with the a sample of teachers as they respond to the various electronic supports in a tune-up. For example, educators always say they want "resources" but do they really use them or just file them?
One more question I have: are the outcomes all self-report or are there performance measures in there too? Thanks.
Jose Felipe Martinez
Thanks for Sharing. I thought your project was very interesting; I found a lot of the premises, assumptions and goals quite compelling. I honestly found it all terrific except for the car tune up analogy...it doesn't work for me because, well people are not cars and the idea is not to postpone the inevitable decay and depreciation that starts the moment they come out of the dealer... :-)
Anyway, I digress, sorry; that's secondary. I am actually very interested in how you went about measuring and/or monitoring the kind/frequency/depth of instructional practices the teachers used. Without a commonly accepted model of NGSS instruction (or instruments to monitor it) this is challenging and a question that we've grappled with in our prior work. Thank you!
Cathy Ringstaff
Senior Research Associate
Hi Jose. Thanks for your comments and questions. In this particular study, we used mostly self-reports, including surveys and interviews. However, we also observed professional development sessions, which provided data on how well teachers were processing what they were learning related to NGSS, and we looked at lesson plans that teachers developed as part of their PD.
Self-reports can be notoriously unreliable, of course. However, we feel that our interviews, observations, and examination of lesson plans helped us triangulate data.
Now that NGSS is being more widely implemented, there are measures out there to help monitor instructional practices. Please email me and I can give you some ideas on what you might use. My colleagues at WestEd have been working on this challenge on various projects.
Jose Felipe Martinez
Thanks a lot Cathy. I used the link and filled out the email form on the WestEd website, but I am not sure if it went through (I received an error message).
At any rate, I am very interested in that line of work; if there is a way you could link to it here that'd be great. Thanks!
Josh Stowers
Channa asked about what this might look like with secondary science teachers rather than elementary teachers. It seems to me that secondary teachers would likely retain higher self-efficacy and content knowledge. I do wonder, however, if professional learning focused on newer teaching strategies (3D science, phenomenon-based teaching, storylines, etc.) in states where NGSS-based standards have been adopted would see a similar decline over time, with teachers slipping back into familiar approaches to teaching. I imagine this type of professional learning would benefit from a similar approach to tune-ups.
Cathy Ringstaff
Senior Research Associate
Hi Josh. I believe that a likely source of elementary teachers' low self-efficacy in teaching science relates to their lack of content knowledge. Many teachers at the elementary level have not had any upper-division science classes, and they worry that their content knowledge is rather shaky. For middle and high school teachers, I agree that they are more likely to retain higher self-efficacy and content knowledge after PL, given that most of them have a stronger science background than their elementary counterparts.
Even if this is the case, I suspect middle and high school teachers would benefit by continued support after the PL, perhaps focusing more heavily on teaching strategies associated with NGSS than on content. NGSS is very difficult for teachers to adapt to, and I am sure "tune-ups" would be welcome. Luckily, there are more and more NGSS resource materials being developed by highly competent experts that we can share with teachers.
The trick is getting the teachers to avail themselves of these resources. At the elementary level, at least, teachers varied in the extent to which they used the resources we gave them. For example, few attended webinars, and few shared via Facebook. The trick is finding which resources are appropriate for teachers in which contexts. There is no "one size fits all" in terms of supports. That's why we suggest that administrators/PL experts offer a menu of supports from which teachers can choose.
Paul Adams
This is very useful research. I am wondering if having short videos would also work for a tune-up that can be used on demand.
Cathy Ringstaff
Senior Research Associate
Hi Paul. Thanks so much for watching our video and leaving a comment. I agree that short videos might be one good way of supporting teachers. There are now a wide variety of videos available online. We offered links to various videos on our newsletters, and teachers seemed to like them, especially if they were short and highly focused on relevant issues.
Paul Adams
Thanks.
Cynthia Crockett
This is fascinating - and wonderful, essential findings! Thank you for your work on this. Ongoing and SUPPORTED PL is essential and often lacking. Love seeing there is evidence behind this approach!
Cathy Ringstaff
Senior Research Associate
Thank you so much, Cynthia! Please take a look at the STEM for All Showcase next year because I believe we will have some preliminary results to share related to our upcoming project in rural Wyoming, North Dakota, California, and Montana. This new project will build on the work described in this video.