True or false? Two students take a quiz with ten questions and each get half the questions wrong. Both have learned and retained the same amount of material from class.
That seems true to most people. But the answer might not be so simple.
If you simply count the the number of correct answers, both students earned the same grade. But that doesn’t mean they’re both in the same place on the learning curve. What if one student got the first half of the questions wrong and found their footing on the last half, while the other student alternated correct answers?
“Traditionally, we just say both of those students got 50 percent,” Dr. Benny Johnson, director of research and development at Acrobatiq, says. “But learning curve analysis recognizes that those 50 percents aren’t equivalent.”
In other words, both students may be struggling, but only one has demonstrated learning.
Learning curve analysis like this lies at the heart of the high-quality adaptive learning. The software doesn’t just take count the answers given by the students. It also observes their behavior during the course, how often they interact with course materials, and how well they do on some questions relative to others. All of these factors combine to provide a detailed analysis of where each student is on the learning curve.
“Underneath the hood, there is some complex statistical simulation that is going on,” Johnson says. “It finds the learning curve that fits the student’s performance in terms of which questions they got right and wrong.”
“Under the hood” in this case means that the learning curve analysis doesn’t get in the way of teaching and learning but is instead presented to instructors and students in a dashboard form.
“Instructors and students don’t need all the information about learning curve analysis,” Johnson says. “They do need to know is how the student is doing so they can concentrate on making adjustments. We want to help people get the benefits of learning science without them having to become experts.”
A brief history of the learning curve
Still, some people feel more comfortable behind the wheel if they understand how the engine works.
To visualize the learning curve, says Johnson, imagine we are workers on an assembly line making Ford automobiles, and we have to learn how to do welding tasks as the parts come past. “Well, of course, the first time we try it, we’re going to be really bad at it,” he says. “We may not be able to do it correctly or if we do, it’ll take us a long time, probably longer than we’d want.”
But we will learn and improve. However, we won’t do so at a constant rate. A learner begins with no knowledge or skill. Once they begin practicing or studying, they gain skills very quickly at the beginning, but the curve evens out and the rise in skill becomes more subtle.
“You start off gaining proficiency quickly, and then the more proficiency you gain, the harder it is to get a little bit more proficiency, so it takes more work to get a smaller gain of proficiency as you get better,” Johnson says.
But the learning curve is more than a path the students ascend as they learn. There are hurdles and setbacks that affect a student’s position on the curve. Adaptive courseware analyzes the learning curve through the lens of those factors.
If a student works on a math lesson today, and later comes back and works on it tomorrow, and again the next week, they’re likely to retain the information. If the student doesn’t work on it for two weeks, however, that student has probably lost much of the information. This learning decay is represented, in Acrobatiq’s model, as an exponential decay that affects a student’s learning.
“There’s a parameter that accounts for how much, on average, we would expect a student to forget or lose in their learning over time, and if it’s been awhile since they’ve touched a lesson, then we just apply that factor to scale them back on the learning curve,” Johnson says.
If it’s only been a day or two since the student last looked at a lesson, it might not matter much to their learning curve. But after a week, the effects become clear. If a student doesn’t touch coursework for two weeks, says Johnson, learning decay could have a substantial impact on the student’s learning curve.
“If they’re being diligent, it really won’t have much of an effect,” he says.
Remember the two students we imagined at the beginning of this post? One answered the test with alternating true-false answers. That’s a pretty good indication of guesswork, but it’s not the only way to tell if a student is guessing.
“Suppose I’m a student doing really badly. I’m just really getting all but the easiest questions wrong, and so I get a bunch of medium or hard questions wrong, that’s my history. Then all of a sudden, you give me a hard question, I get it right, and then I go back to getting them wrong again,” says Johnson.
By taking a student’s past performance into account and using it as a context for their current performance, Acrobatiq can determine whether a student is guessing, giving an instructor a clearer picture of where that student is on the learning curve.
Other factors may affect learning
As Acrobatiq’s software evolves, other variables may be included in the courseware’s learning curve analysis. These factors may include students’ confidence about their learning, whether they are goal-oriented or performance oriented, and their beliefs about learning in general.
Students who believe that their intelligence is fixed, says Johnson, may not learn as effectively as students who believe they can improve their intelligence through practice.
“The research shows that, of course, our intelligence is not entirely fixed,” he says. “We can do things to improve it, and so variables like that may also be useful to include in setting the learning curve.”
A.J. O’Connell is a freelance journalist specializing in education reporting and a former college journalism teacher.