Its purpose is to support student learning, but Canvas, Indiana University's online learning-management system, is helping academic advisors learn something of value too. Using data from Canvas to identify students who may need academic support, IUPUI Campus Career and Advising Services has designed an early-intervention program that's improving student success.
The intervention, launched last fall as a yearlong pilot study in University College, is based on activity and engagement in Canvas—behaviors that often correlate to student performance.
Capturing student-specific data across all of their courses, including number of assignments due, number of submitted assignments and overall minutes spent in Canvas, the advising team calculates a "Canvas engagement score" for each student. Individuals with scores falling within the lowest quartile for engagement in their school or unit are contacted by their academic advisor for a check-in.
Scores are calculated in two-week intervals during the first six to eight weeks of the semester, offering three opportunities for students matching the criteria to be contacted.
"Students receiving the advising intervention last fall had a term GPA that was on average 20 percent higher than our control group, and their persistence rate at the institution was higher as well," said Matt Rust, senior executive director of Campus Career and Advising Services. "Our goal was to determine whether this intervention is effective, and we've certainly seen enough success in this first semester to scale up the effort, not only in University College, but also in sharing the results with other IUPUI schools and IU campuses."
While the pilot study shows there is significant predictive value tied to Canvas engagement, Rust said looking exclusively at engagement in the online system inherently results in a few "false positives"—flagging students, for example, who are on track academically, but not regular users of Canvas. Because the data capture doesn't include grades and doesn't account for optional assignments, the formula for identifying the "right" students for the intervention isn't foolproof. Even with these limitations, the false-positive rate was below 5 percent for the fall semester.
"We expect to make some modifications for outreach based on what we learn, but we don't need it to be perfectly predictive," Rust said. "We just want to get an idea, based on some reliable engagement criteria, of who we should be checking in with during the opening weeks of the semester."
In fact, Rust said, even in cases when students who don't need the Canvas intervention have been contacted, advisors have been able to make valuable connections that could lead to future interactions.
"Reshaping the understanding of advising is definitely a big part of this," he said. "With a more proactive approach, advising becomes a comprehensive set of conversations that focus not just on academic planning and policies, but career planning, engagement in high-impact practices, use of campus resources, and intellectual and personal development."