R305A180211
This project explores factors related to student success in online STEM college courses, and especially factors that might be leveraged to support students from groups that are often underrepresented in STEM domains (UR-STEM students). In one line of research, we focus on help-seeking behaviors in course discussion forums. We expected that the relatively anonymous nature of online discussion forums would promote help-seeking behaviors from UR-STEM students who might otherwise feel excluded or stereotyped in a traditional course. The results support this idea, and suggest that UR-STEM students may be even more likely than their peers to engage in help-seeking in such an environment.
In a second line of research, we explore issues of help-giving in online STEM course discussion forums. That is, when someone actively seeks help, what aspects of a response explain the perceived helpfulness of that response? We surveyed students regarding their perceptions of example requests for help and responses to help, sampled from previous discussion forum interactions. We then analyzed these responses to construct a model for assessing the helpfulness of students’ responses to their peers. In future work, we plan to teach students to provide more helpful responses based on this model, and thus to leverage the inclusive nature of course discussion forums to improve learning for all students.
Nigel Bosch
Assistant Professor
Hi, everybody,
Thank you for checking out our video! This is an exploratory project studying some of the ways in which students from demographic groups traditionally underrepresented in STEM utilize typical online learning spaces in college courses. We are in the last year of the project, and have focused recently on discussion forum interactions, which prove to be valuable means of engaging with peers in a learning environment that can be somewhat disconnected by nature. As we look to the future, we're especially thinking about how to teach students to make the best possible use of these forums -- both in terms of asking for help, and providing effective, valued help.
We welcome and questions or feedback you have! We're especially interested in future work building on discussion forum research for college-level STEM courses, including, for example, ideas to improve the value of forums for students. How are forums positioned to make them a valued resource rather than a task for students to check off the list? What types of discussion formats encourage the most effective help-seeking/giving? We are excited to hear any thoughts, ideas, and experiences!
Michelle Perry
Jeff Milbourne
STEM Coordinator-Writing and Learning Center
@Nigel Thanks for the work, and the video. I think this comment/question, 'how are forums positioned to make them a valued resource rather than a task for students to check off the list?' really hits the nail on the head. Anecdotally, I've struggled with how best to leverage discussion forums as a learning space for years (as both a student in an online course and an instructor of an online course), so I appreciate that focus. You mentioned a conceptual model in your description; do you have any references you can share that dig into the results a bit more? I know a 3' video is somewhat constraining in terms of the depth/detail you can provide, but I am curious to know more about the specifics around some of the results, particularly from the online chemistry course.
Michelle Perry
Nigel Bosch
Assistant Professor
Hi Jeff,
Certainly! Here is a link to a preprint of the article from the chemistry education context: https://publish.illinois.edu/ilearngroup/files/2022/02/Ask_for_help_manuscript_final.pdf
In some cases -- like this chemistry course -- there are many situations where students get real value out of the discussions, and other times (like my own courses, it seems!) students don't see it that way. Anecdotally as well, I have enough clarifications and questions from students through other channels (mostly email) in online courses to suggest there is a need for the kind of support a discussion forum can provide. But, perhaps sometimes the incentives and risks (e.g., fear of seeming unknowledgable to peers) of forum participation make it seem a bit less attractive of an option.
Michelle Perry
Jeff Milbourne
STEM Coordinator-Writing and Learning Center
Thanks for the reference, and I agree that the public/private nature of online discussions can impact the nature of those discussions.
I often struggle with how to encourage/invite vulnerability in public spaces that are online, which is part of a larger challenge I experience when trying to create community in online courses. It's always nice to see examples of that sort of thing done well, which it seems the online chemistry course did. I look forward to reading over your manuscript.
Michelle Perry
Michelle Perry
Professor
Hi Jeff,
In related work, we have found that the structure of the forum can play a role for students in participating and in asking questions. For example, in one course that we examined, participation in the forum was worth 25% of students' grades and students were required both to post a question and answer another student's question in the forum each week. Not surprisingly, this requirement yielded significant participation and interaction among students in the forum.
In some ways, we expect that this can serve to level the playing field (i.e., ALL students ask questions) and students have the potential to form a culture where they authentically seek information they need (e.g., when their questions get treated with respect and they receive answers that are helpful). Understanding how to create this culture is clearly something worthy of understanding and pursuing, which is one direction we're planning to pursue.
Michelle
Michelle Perry
Sarah Haavind
Jeff Milbourne
STEM Coordinator-Writing and Learning Center
Thanks Michelle. Yes, that does make a lot of sense in terms of structuring course incentives to encourage participation and level the playing field. I suppose my next question is how to ensure quality participation (as opposed to just checking the requirement)-and, for the record, this is more a function of my own struggles to encourage quality discussion in online forums...
Michelle Perry
Destiny Williams-Dobosz
Sarah Haavind
Senior Research Project Manager
Hello University of Illinois Urbana-Champaign project team and Jeff! This research project is of special interest to me, as are approaches generally to deepening online learning. This thread of conversation is particularly of interest as, like Jeff (and probably most visitors here) I'm always thinking pedagogically of ways to ensure quality participation - and I'm intrigued by the approach of giving 25% credit for posting questions and responses weekly. As professors, we all want to make sure we aren't answering questions to individual students via email that all could be learning from (or worse, over and over writing similar responses, managing private email!). A starting point would be to have a specific forum for "Questions Related to Problem-Sets" or other homework help-oriented open forum for Q&A. Even if a student is spending time responding - and thereby writing about what they presumably already know, one could argue "quality participation" due to the cognitive process of crafting a clear, succinct answer that another student could understand. A lot of productive cognitive engagement can go on right there. Giving so much of the course grade to Q&A activity also simplifies setting up a culture of safety - I'm asking because the instructor requires questions and I'm answering because that's required as well. Very safe. I actually think it's pretty interesting! I am delighted to see the team is already planning to pursue a better understanding of effective ways to create a collaborative culture of helping in online environments. Making the behavior count is a winner.
Michelle Perry
Destiny Williams-Dobosz
Nigel Bosch
Assistant Professor
Personally I have those same experiences quite often as well -- sometimes a student asks a wonderful question in email, and I wish everyone saw it! Or, I get the same procedural question several times (especially, this time of year, when will grades be posted?).
Sarah Haavind
Michelle Perry
Destiny Williams-Dobosz
Graduate Research Assistant
Hi everyone,
Great discussion! I echo everything that Michelle and Nigel have said. In addition, offering credit for posting to the forum, and getting students to engage with the forum early in the semester is key. Often, students don't view forums as a useful learning resource or forget to use them altogether.
In terms of quality participation, there are many sides to consider. I agree that different forums serve different purposes, and there are various ways to implement them.
And yes, we do see productive cognitive engagement in the forums, in this paper, you'll see examples of students writing detailed answers in a supportive fashion.
https://publish.illinois.edu/ilearngroup/files/2022/02/Ask_for_help_manuscript_final.pdf
In related work, we are currently investigating how to streamline discussion threads through writing prompts. This more so speaks to general advice and community building to explore further a classroom culture of safety.
Sarah Haavind
Michelle Perry
Jena Barchas-Lichtenstein
This is a really neat project. I'm curious to hear more about methods - was there a discourse analysis component? What form did that take?
Michelle Perry
Destiny Williams-Dobosz
Graduate Research Assistant
Hi Jena,
Thanks for this question. For the "Help-Seeking in Online Courses" project, we analyzed help-seeking posts (i.e., requests for help) across the level of explicitness. Then, we analyzed the subsequent responses to an initial post in terms of how substantive and detailed the answers were, so it was more of a contextual analysis. The methods are outlined in this paper, https://publish.illinois.edu/ilearngroup/files/2022/02/Ask_for_help_manuscript_final.pdf
Michelle Perry
Michelle Perry
Professor
I also checked in with one of our co-authors, Dr. Genevieve Henricks, who adds this insight about some discourse analytic techniques she used, to examine whether men and women used gendered language in the discussion forums. She wrote:
We conducted a text analysis to explore the use of gendered language in the discussion forums. To do this we used Linguistic Inquiry and Word Count 2015 (LIWC; Pennebaker et al., 2015), the 2001 version of which Newman et al. (2008) also used to analyze gendered language. We used their study as a building block for choosing which LIWC categories to examine for possible relations to use gendered language, and we ultimately examined the LIWC counts for Word Count; ratings for the Authentic, Analytic, and Clout categories; and percentages for the Pronouns, Discrepancies, Tentative, Social Processes, Numbers, and Certainty categories. We further explored whether the gendered language varied according to posting type (e.g., posting a solution to a homework problem, asking a question, or answering someone’s else's question). We found that although students used gendered language, they did NOT do so along gendered lines; however, we found that posting types revealed differences in gendered language use.
Amos Jeng
Graduate Research Assistant
Hi Jena,
Thank you for the question! I can speak to the methods used in the "Help-Giving in Online Courses" component of this work. For that study, we employed a grounded theory approach to qualitative analysis because our goal was to develop a model of online help-giving that identifies characteristics of helpful online discussion forum replies, based on students’ perspectives. As the video mentions, we had participants complete a survey where they evaluated the helpfulness of various examples of replies to requests for help posted to a college course discussion forum. Specifically, participants were instructed to rate the helpfulness of each example on a scale from "Not helpful" to "Very helpful," as well as explain why they selected the level of helpfulness they did via an open-ended text response. These written open-ended responses provided the input for analysis, and through an iterative coding process, we identified message characteristics cited by participants as positively contributing to the helpfulness of online replies.
Sarah Haavind
Michelle Perry
Sarah Haavind
Senior Research Project Manager
Hello again Dr. Bosch and team! Fun project and nice discussion, thank you! I'm noticing you mention in your description that your focus is those "factors that might be leveraged to support students from groups that are often underrepresented in STEM domains" and your thought is that the anonymity of online venues might help encourage active participation. Can you tell us more about how your participants were selected? Thanks in advance for your thoughts.
Michelle Perry
Nigel Bosch
Assistant Professor
Excellent question! We have two approaches represented here: one is a large, retrospective study of all students across several sections of different courses, which is the source for Destiny's work discussed in the video. We focused on introductory STEM courses that might be perceived as "gateway" courses to STEM participation, which may be barriers to graduating with a STEM degree. For Amos's work, participants were college students with appropriate familiarity with the topic, so that they could judge the quality of helpful replies effectively.
Sarah Haavind
Michelle Perry
Bhaskar Upadhyay
Associate Professor
Online is the new norm of teaching and learning. Discussions are key to learning but what is concerning is that the same students take the initiative and the rest get lost through the cracks or do not find it comfortable to participate in this environment. What is not clear is how the students "from underrepresented groups encouraged and supported to participate in the discussion"; "how do the instructors keep track of who and why they participate or not in the discussions?" I like the efforts in this group has put in.
Michelle Perry
Michelle Perry
Professor
Adding to Destiny's important points, across the courses we've examined, in different STEM fields, we've found positive effects for behaviors we'd expect to impact learning (e.g., engaging in self-regulated learning strategies, such as metacognition), for students from all demographic groups.
As an example, Nigel Bosch and colleagues (https://pnigel.com/papers/bosch-urimprove-umap1...) noted that students’ behaviors, including exam submission times and forum usage, were related to improvement, independent of whether they were from groups underrepresented in STEM or not. This suggests that online STEM courses do not disproportionately disadvantage students from groups underrepresented in STEM and that encouraging students to engage in behaviors that should support learning may be especially crucial to consider for online college STEM courses.
Sarah Haavind
Destiny Williams-Dobosz
Graduate Research Assistant
Great questions! I can speak for the chemistry education project, which primarily utilizes retrospective analyses to study online behavior patterns. So, instructors operate their courses as usual, and then we look for patterns in the data. Requiring students to participate in the forums for a portion of their grade ensures that all students use the forum, especially those from underrepresented groups.
In one study, a social network analysis, we saw that those from underrepresented groups were highly active in the course forums - but our future work will explore these further in more STEM domains and course levels. We suspect that this finding may not generalize to all online course contexts, thus, warranting more investigations.
doi.org/10.1145/3448139.3448159
Furthermore, we are looking to incorporate multiple methods and mixing methods (e.g., surveys, interviews) to examine why students (especially those from underrepresented groups) do or don't participate in the forums. Another interesting part of this work is that we use the research findings to help instructors understand how students engage or disengage in the forums and where some are getting lost in the cracks - thus, addressing where modifications and recommendations can be made.
Sarah Haavind
Michelle Perry
Jason Morphew
I really enjoyed this video. I think that everyone is doing great work on this project. In my experience as someone who teaches large sections of introductory STEM students, promoting asking for help is critical for struggling students. In engineering courses there is often a culture where students want to figure things out for themselves as a way to prove that they belong. Unfortunately, it is the struggling students who often fall into this trap (maybe because they are already struggling and need to prove themselves more). I am wondering if you have found any differences between STEM majors and STEM related students (those who have to take Chemistry for example like pre-med students)? Another thing that made a connection for me was the possibility that help-seeking may be related to academic goal orientations.
Sarah Haavind
Michelle Perry
Professor
Hi Jason,
Thank you for raising these interesting points. Starting with your second point, we did not examine relations between academic goal orientations and help-seeking, given the nature of the data (for most of this project, we conducted analyses after the course was completed and did not intervene with the students while they were taking the course).
In some courses we've examined, we have looked at whether the student was a STEM major and, in all cases that I can recall, we have NOT found any relation between STEM major and the behavior of interest (in this case, asking for help in the discussion forum).
Still, I think you're onto something important with these two points. In particular, we should be digging deeper to see what might prevent some students from seeking help when they need it. Then, once we can identify what is deterring them, we can begin to find ways to help them avoid the deterrent and encourage them to seek help when needed.
Sarah Haavind