1. Rachel Dickler
  2. Research Associate
  3. NSF AI Institute for Student AI Teaming
  4. https://www.colorado.edu/research/ai-institute/
  5. CU NSF AI Institute for Student AI Teaming, University of Colorado Boulder
  1. Indrani Dey
  2. PhD Student
  3. NSF AI Institute for Student AI Teaming
  4. https://www.colorado.edu/research/ai-institute/
  5. University of Wisconsin Madison, CU NSF AI Institute for Student AI Teaming
  1. Shiran Dudy
  2. http://www.shirandudy.com
  3. Research associate
  4. NSF AI Institute for Student AI Teaming
  5. https://www.colorado.edu/research/ai-institute/
  6. University of Colorado Boulder, CU NSF AI Institute for Student AI Teaming
  1. Julie Harrison
  2. PhD Student
  3. NSF AI Institute for Student AI Teaming
  4. https://www.colorado.edu/research/ai-institute/
  5. Georgia Institute of Technology, CU NSF AI Institute for Student AI Teaming
  1. Rachel Lieber
  2. Research Administrative Professional
  3. NSF AI Institute for Student AI Teaming
  4. https://www.colorado.edu/research/ai-institute/
  5. CU NSF AI Institute for Student AI Teaming, University of Colorado Boulder
Public Discussion

Continue the discussion of this presentation on the Multiplex. Go to Multiplex

  • Icon for: Dorothy Bennett

    Dorothy Bennett

    Director of Creative Pedagogy
    May 9, 2022 | 03:43 p.m.

    Hello,  Was wondering how students related and reacted to these robots intervening in their group work?  What roles did teachers find themselves playing? Thanks!

     

  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 9, 2022 | 04:09 p.m.

    Hello Dorothy! Thank you so much for your questions! We are just starting to take our next steps towards working with students to get their feedback on the implementation of the AI agent in their group. The exciting part about this work is that we are still deciding the form the agent will take (e.g., a robot embodiment, a disembodied voice, an avatar on a computer screen, a computer display, or other option). It will definitely be fun to see students' reactions to the robot embodiments and how the robots impact the students' experiences! 

    The AI agent will augment teacher instruction, but teachers will continue doing the amazing work that they are already doing in classrooms! Depending on the role of the agent, it may be able to help students with collaboration and understanding of content as well as display data to teachers. We aim for teachers to continue in their roles while the agent best supports teachers' particular needs. Our upcoming participatory design work with teachers will be critical for helping us to further define the exact role of the agent. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Pati Ruiz
  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 9, 2022 | 03:52 p.m.

    Welcome to the NSF National AI Institute for Student-AI Teaming (iSAT) video page! iSAT is imagining how AI-agents can help teachers and students during collaborative group work in classrooms. We are excited to read and respond to your questions and comments. We are particularly interested in your thoughts on the design sprint process and resulting AI-agent roles. We also welcome general discussions on the potential role AI-agents may play to support collaborative learning in classrooms. 

  • Icon for: Dorothy Bennett

    Dorothy Bennett

    Director of Creative Pedagogy
    May 9, 2022 | 05:12 p.m.

    Sounds interesting Rachel.  I am curious to hear what you learn from the participatory design work.  The role of agents in the classroom, I think, really needs to consider how teachers think of their role in instruction and what learning it can enhance. 

  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 10, 2022 | 12:20 p.m.

    Thank you so much Dorothy, yes we completely agree! We look forward to sharing our results!

  • Icon for: Chris Dede

    Chris Dede

    Higher Ed Faculty
    May 10, 2022 | 07:48 a.m.

    Rachel, I'm glad to see your National AI Institute is participating in the STEM Videohall. Please take a look at our video: https://stemforall2022.videohall.com/presentations/2364  It would be great to find ways to collaborate

     

  • Icon for: Julie Harrison

    Julie Harrison

    Co-Presenter
    PhD Student
    May 10, 2022 | 10:23 a.m.

    Chris, I am one of the co-presenters of this video--one of the Georgia Tech collaborators. I attended a talk you gave at GT a few months back. Your concept of teaching 'judgment' skills vs. 'reckoning' abilities has stuck with me and I think it is very applicable to the work we are doing at iSAT to teach and support collaborative problem solving. Glad to see another NSF AI institute represented--looking forward to seeing how your institute evolves. Thanks for stopping by our page.

     
    1
    Discussion is closed. Upvoting is no longer available

    Pati Ruiz
  • Icon for: Chris Dede

    Chris Dede

    Higher Ed Faculty
    May 11, 2022 | 07:07 a.m.

    Happy to stay in touch about this issue. Perhaps we can have a cross-institute discussion about it.

  • Icon for: Dalila Dragnic-Cindric

    Dalila Dragnic-Cindric

    Researcher
    May 10, 2022 | 02:13 p.m.

    Congratulations on this exciting work! In this video, Rachel Lieber mentioned exploring metaphors used to describe AI's role in classrooms. Can you share a bit more about what specific metaphors were being used, and did you see a difference in who was using which metaphor? For example, did you find that teachers and students were using different metaphors?

  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 10, 2022 | 04:27 p.m.

    Thank you so much Dalila! That's an excellent question! We started off the design sprint with several different ideas for how an AI agent could support teachers and students, and then narrowed our focus to three specific metaphors:

    • Community Builder: The Community Builder would be an AI Agent to support students in developing trusting collaborative relationships with each other. The Community Builder would uplift non-normative forms of collaboration to create more equitable collaborative spaces. 
    • Co-Pilot: The Co-Pilot would be an AI Agent to support collaborative problem solving through promoting discussions within and across small groups. The Co-Pilot would primarily focus on supporting domain knowledge (e.g., helping students understand how to block program).
    • Augmenter: The Augmentor would be an AI Agent to support teachers and students by providing data through a visual display. The Augmentor could help to relay information between teachers and students as well as guide collaboration (i.e., augmenting teacher instruction). 

    We had groups of undergraduate students, graduate students, post-doctoral researchers, and early career scholars who participated in the design sprint event to create these metaphors. The designs of the metaphors were heavily influenced by feedback we had received from teachers and students including through the iSAT Learning Futures Workshops. We are looking forward to bringing these ideas back to teachers and students for feedback in the upcoming months! It will definitely be interesting to see which components are most helpful to different teachers and students depending on their specific needs.

  • Icon for: Judi Fusco

    Judi Fusco

    Researcher
    May 10, 2022 | 08:15 p.m.

    This is great.  I look forward to talking more and we are  happy to share anything you want on Educator CIRCLS. (We'll tweet about your video, too!) We think a lot about how AI systems and teachers might partner.  And of course, relevant for the AI Engage Institute, too. 

     
    1
    Discussion is closed. Upvoting is no longer available

    Rachel Dickler
  • Icon for: Chris Dede

    Chris Dede

    Higher Ed Faculty
    May 11, 2022 | 07:05 a.m.

    I hope we can build collaboration among the three Institutes focused on education. Some of the AI-agents we are building relate to community building.

     
    3
    Discussion is closed. Upvoting is no longer available

    Rachel Dickler
    Julie Harrison
    Judi Fusco
  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 11, 2022 | 12:21 p.m.

    Thank you so much Judi and Chris! Yes that would be great!

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Icon for: Nathan Holbert

    Nathan Holbert

    Facilitator
    Associate Professor
    May 10, 2022 | 03:06 p.m.

    Love this idea of using AI to support collaboration between not just machines and humans, but also between humans and humans! As a research community our initial efforts around AI in education is aimed at building an answer machine. But of course we know the best teaching isn't just having the right answers, it's supporting learners in a complex process!

    About the design sprint. Were there any surprises that emerged or any insights from a group or team that suggested some unexpected but exciting directions for AI?

  • Icon for: Rachel Dickler

    Rachel Dickler

    Lead Presenter
    Research Associate
    May 10, 2022 | 04:44 p.m.

    Hi Nathan! Thank you so much, we are enjoying deeply exploring the ways in which AI can support collaboration between humans (particularly between middle and high school students)! 

    One critical insight that emerged from the design sprint was the idea of using AI to support “non-normative” forms of collaboration (as part of the Community Builder metaphor for an AI Agent). Non-normative forms of collaboration include behaviors that are commonly discouraged in classrooms or are perceived as unproductive (e.g., “off-task” discussions). Often, however, these non-normative behaviors make for more equitable collaborative dynamics as well as present an opportunity for building relationships between students. This brings to the forefront exciting directions for AI in terms of being able to capture and support the variety of normative and non-normative ways in which students collaborate. This will likely require AI models that triangulate speech with gesture, posture, eye-movements, and other forms of multi-modal data to understand the different types of collaboration that are occurring across classroom settings (which can be particularly challenging to capture given the amount of movement and noise during group work). We are looking forward to pushing the boundaries in this area to see how we could best support our teachers and students!

     
    1
    Discussion is closed. Upvoting is no longer available

    Nathan Holbert
  • Icon for: Joshua Danish

    Joshua Danish

    Facilitator
    Professor and Program Chair
    May 11, 2022 | 04:24 p.m.

    Ooh, this is a really nice angle. I wonder if you can say more about how you'll identify when normative or non-normative behaviors are "good"?  Is that something you imagine the teacher might help with? 

  • Icon for: Julie Harrison

    Julie Harrison

    Co-Presenter
    PhD Student
    May 12, 2022 | 10:48 a.m.

    From a measurement perspective, we are are working to model CPS dynamics in real time. One example we discuss is with off-task/topic conversation. Sometimes this off-topic conversation is a tool students use to become integrated with group members. It first appears unproductive to CPS, but it is actually contributing to group member inclusion and group cohesion. We are working to model this communication behavior over time in order to assess what types of non-normative CPS communication tend to benefit the group during different phases of collaboration. 

    Rachel or Indrani can speak to ways the teacher might help with this with regard to the teacher dashboards. 

  • Icon for: Indrani Dey

    Indrani Dey

    Co-Presenter
    PhD Student
    May 12, 2022 | 11:10 a.m.

    Yes, teachers will be critical to this component! We have conducted participatory design sessions with teachers to better understand their dashboard needs in the classroom, which we will use to inform our designs. We are also in the process of considering how we can integrate teacher feedback within a dashboard to inform our AI models, particularly for complex areas that vary or may look different across classrooms.

  • Icon for: Nonye Alozie

    Nonye Alozie

    Researcher
    May 10, 2022 | 11:21 p.m.

    This is great. Thanks for your video. I met Arthur Graesser a couple of weeks ago- he and I sit on the advisory board for the AI ALOE Institute. He mentioned that your group was also studying collaboration. I look forward to seeing more about the role that an AI collaborator plays in student collaboration. What are you expecting to see, in terms of the role of the AI partner?

     
    1
    Discussion is closed. Upvoting is no longer available

    Judi Fusco
  • Icon for: Julie Harrison

    Julie Harrison

    Co-Presenter
    PhD Student
    May 11, 2022 | 10:50 a.m.

    As the institute grows, we will be and currently are developing the role of the agent alongside young people/students and teachers. Currently, the students in the Learning Futures Workshops are generating their own creative ideas for the parter (e.g., a simple light up display that monitors collaboration norms--somewhat of an artifact of how collaboration is going to ground the whole group). The role of community builder has also resonated with them. When interacting with teachers, we are currently focused on what they would like to see on both after-action review dashboards and in-real-time dashboards. We are working with teachers to ensure the dashboards are customizable and allow teachers to engage fully with the class during class time (i.e., design in ways that prevent them from having to look only at their iPads during class time).

    Arthur Graesser's work with CPS measurement has been influential to the skills we are looking to measure with the iSAT CPS measurement framework. The work presented in your video is also relevant to the iSAT framework. One of our goals is to build out more affective skills/behaviors that contribute to successful CPS -- I will be adding confidence to that body of skills after seeing your video!

  • Icon for: Joshua Danish

    Joshua Danish

    Facilitator
    Professor and Program Chair
    May 11, 2022 | 04:25 p.m.

    This is really wonderful work - thanks for sharing! I look forward to hearing more about it! One question I had was: how are you thinking about diversity amongst your learners? Different learners naturally have different needs in collaboration. Are you working with diverse groups for your ongoing participatory design work? Or is that something you'll aim to adapt to later? I really appreciated the note above about non-normative forms of participation. Thanks!

  • Icon for: Julie Harrison

    Julie Harrison

    Co-Presenter
    PhD Student
    May 12, 2022 | 11:11 a.m.

    Hi Joshua, thanks for your comment. We are certainly considering diversity -- along social and economic dimensions, as well as learning and interaction preferences for students. There are a couple hurdles we are working to address. 1) Much of the collaboration measurement literature is focused on drawing insights from students' communications (e.g., transcripts). We recognize communication is not every student's contribution to a group, so we are working on multimodal measurement, and will now also look into ideas we've seen in other presenters' STEM for All videos for this, as well. 2) The data we are collecting from schools tends to have a bias regarding those students (parents) who opt into our data collection efforts. Generally, those who skew higher on levels of socioeconomic status are more likely to be willing to participate in data collection. In forthcoming data collection efforts, this is a priority that we will address.

  • Icon for: Josh Sheldon

    Josh Sheldon

    Facilitator
    Project Lead
    May 12, 2022 | 04:44 p.m.

    This is really interesting work - thanks so much for sharing.

    There's been quite a bit of good discussion already. I'll try to follow up on some points and perhaps unearth a few others:

    • Re: non-normative discussion & collaboration, this doesn't have to be something that an AI agent is uniquely good at. Humans can certainly encourage non-normative (or what might be considered by others to be non-normative) interactions. I recently read a piece by bell hooks (I believe it was the first chapter of Teaching to Transgress) in which she made a strong case for thee absolute need for teachers to engage in what we're calling non-normative interactions to know and reach learners. So the question, for anyone involved in this discussion is why a human or AI might be better at encouraging these behaviors?
    • Along similar lines, would we choose an AI agent over a human para-professional to be in a class to reach the collaboration (and other) milestones with learners? Why and how would one or the other be desireable?
    • I'm excited to hear that learners (MS & HS) are being engaged in participatory design. My experience is that it's great to bring them in as early as possible. How are you working to gather as many perspectives as possible from the many intersecting identities of learners that make up our country, and if a decision needs to be made about what perspective to center, how do you think about that?

    Looking forward to hearing more!

  • Icon for: Indrani Dey

    Indrani Dey

    Co-Presenter
    PhD Student
    May 17, 2022 | 12:59 p.m.

    Hi Josh, thanks for your great questions! 

    • There are often multiple activities going on in the classroom at any point in time, with the teacher only able to support one individual or group at a time. One of our goals is for the AI agent to be able to offload some of the support so that the teachers can more easily facilitate or help students who need their help the most. This may include (but not be limited to) recognizing inequitable participation (which the teacher may not have noticed in one group while they are helping another) and encouraging more collaborative behaviors.
    • Our intention is not to have the AI agent replace a human in the classroom, but rather, be a complementary tool to the teacher / staff support. One teacher is not always able to reach multiple students or groups simultaneously, which is where the AI would have an advantage if a teacher is already busy helping a particular group. However, the teacher will also have access to group collaboration metrics and using human-in-the-loop features, be able to better train the AI when to intervene.
    • We are working very closely with teachers and students (currently across the US) through events such as the Learning Futures workshop that are grounded in co-design. This helps us to iteratively build and redesign based on what we learn from our students, from their past and current perspectives (e.g., collaboration experiences in the classroom), and their visions of ideal or desirable environment(s). A key component of our design process is considering customizable features that allow for addressing the wide range of different specific needs across our users.
  • Icon for: Josh Sheldon

    Josh Sheldon

    Facilitator
    Project Lead
    May 17, 2022 | 01:14 p.m.

    Hi Indrani.

    Thanks for your response.

    I understand that the agent is not to replace a human. So, assuming the lead teacher in the classroom stays the same, my curiosity is around the value of an AI agent vs. a second trained human, either a less-experienced co-teacher or a paraprofessional. And of course, I was mainly curious about the costs & benefits around educational and social outcomes, recognizing that there are huge practical implications, financial and otherwise, but choosing to ignore those for now.