Can AI-powered virtual humans help students learn how to learn and develop SRL skills? This was the focus of a research project jointly conducted by Edge – Center for EdTech Research and Innovation, Temasek Polytechnic, Oranim Academic College and Beyond Virtual. The findings suggest that students who completed the virtual-human training have adopted the cognitive learning strategies recommended by the video-bot.
The research study was conducted over 5 months from (April – August 2023). The research project was led by Dr. Danny Glick – Academic Director at Edge – Center for EdTech Research and Innovation and Research Affiliate at UC Irvine, Dr. Huiyu Zhang – Senior Lecturer at Temasek Polytechnic in Singapore, Dr. Shirley Miedijensky – Associate Professor at Oranim Academic College, and Beyond Virtual. Then, the findings were presented at the WCET Annual Meeting (Western Interstate Cooperative for Educational Technologies) in New Orleans in October.
When traditional training interventions don’t do the trick
Students who struggle to self-regulate underperform compared to students who self-regulate their learning. The outbreak of COVID-19 accelerated the adoption of online learning and hybrid tools. As a result, schools were shifting from traditional learning environments to online and hybrid learning models, where students must be able to self-regulate in order to succeed. However, many students struggle to self-regulate, which further widens the gap between students who know how to self-regulate and those who don’t. This often leads to huge learning losses, which may lead to a learning and skills crisis.
Research shows that this lack of attention to helping students learn how to learn becomes evident when students encounter challenging coursework in university settings – and are expected to manage their own learning independently. Many students who never learned those skills in high school struggle in their STEM coursework. When they underperform, they drive the well-documented STEM retention problem. Therefore, there is a pressing need for personalized, flexible, and scalable training programs to train first-year university students to learn how to learn.
A wide range of traditional SRL training programs have been developed and implemented in recent years. However, traditional SRL training programs are limited. To start with, they rely on face-to-face interaction with facilitators who supply considerable amounts of individualized instruction and feedback to students, which makes traditional training non-scalable and high in cost. Additionally, they require large investments of time and effort by both students and instructors. Finally, more often than not, they require more extensive time investments than those pursuing undergraduate STEM degrees can commit to. And this is when Beyond Virtual comes into play.
Other use cases of Beyond Virtual:
- How to use “UnHeard” for language learning
- How we boosted junior-high students’ self-confidence
- How we helped college students improve their presentation skills
Self-paced SRL Training
The current study employed a two-group quasi-experimental design, in order to investigate the effects of different self-regulated learning (SRL) training interventions on students learning behavior. The study involved 100 students in the treatment group who had access to virtual-human SRL training. At the same time, 50 students in the control group were provided with a digital SRL training booklet. The treatment group had the flexibility to complete the SRL Virtual-Human training at their own pace. Throughout the study, behavioral variables representing SRL activity were observed. They included planning, goal setting, self-monitoring, and reflection.
A mixed-methods design – which mixed both quantitative and qualitative research – was adopted. The quantitative analysis focused on independent variables like the portion of training completed and interaction turns. Dependent variables included SRL Click Tracking (traces of SRL learning behavior observed through MS Planner log files) and changes in pre- and post-self-reported student questionnaires.
For the qualitative research, we conducted focus group interviews to assess various aspects of the training. Among them were bot design, functionality, accuracy, user-friendliness, and conversation flow. These interviews involved three distinct groups of participants. First, we interviewed treatment-group students who completed the virtual-human training. Then we met the treatment-group students who did not complete the virtual-human training. Finally, we reached out to the control-group students.
“It really helped me stay on task”
The research’s results validated our hypotheses. We found that completion of the first training module (Planning) had a positive effect on student use of Planning tools within the MS Planner environment. As for monitoring, the results were similarly encouraging; We found that a high level of user-bot interaction had a positive effect on student use of Monitoring tools. No effect was found for completion of the third training module – Reflection – and the use of Reflection tools within the learning environment.
The focus group interviews suggest that students had a positive experience when using the virtual humans. “It definitely helped us. We had to make sure that we weren’t missing any important information”, one of them said. Others defined the virtual human as “useful”, saying that “It really helped me stay on task”. Furthermore, the student said: “If I hadn’t used it, I would have missed some of the features.”. The students and their mentors gave us valuable feedback for future development – both for SRL purposes and for other use cases in EdTech.
- Dr. Danny Glick contributed to this article