https://early.khanacademy.org/open-ended/

On Khan Academy and other online learning platforms, students build skills by practicing with interactive exercises that deliver instant feedback. While these activities work well for factual and procedural knowledge, students also need to learn complex reasoning skills, including: literary analysis, scientific inquiry, critical thinking, mathematical modeling. To develop these skills, students need practice explaining and justifying their ideas. They also need feedback beyond a grade; they need specific, actionable feedback to guide their revision and iteration.

The Early Product Development group at Khan Academy began investigating this question in early 2017. Since then, we’ve worked with teachers across the country to run classroom pilots and to iterate on our solution.

We’ve created a prototype learning platform that offers students rapid feedback by choreographing an intricate dance between students, their respective ideas, expert-authored model work, and lightweight teacher facilitation. There’s still much to do, but now that our latest pilots have received glowing reviews from teachers and students, we’re thrilled to share our approach.

“Why is there a big spike in our data on this plot?”

“Since globalization, what’s changed? What’s stayed the same?”

“What are the most important causes of structural inequality?”

“How does water quality affect fish populations?”

The most challenging and meaningful questions have no straightforward answers. They don’t come with a simple set of rules that always yields a solution. These open-ended problems are “ill-defined,”

and answering them requires complex reasoning skills: skills that build on extended inquiry, creativity, and critical thinking.

Such skills are part of every core subject’s curricular standards, but exam results indicate students struggle to develop these complex reasoning skills.

• Math students must develop modeling skills like “choosing and using appropriate mathematics and statistics … to improve decisions.” But 41% of California’s 11th graders could not reliably demonstrate this type of skill on 2017’s state test.
• Science students must “construct their own explanations of phenomena.” But 58% of students got 0/7 points on this type of task on 2018’s AP Physics 1 test.
• Social studies students must “assess authors’ claims, reasoning, and evidence.” But in the 2017 AP US History exam, 49% of students failed, with this skill cited as a common knowledge gap.

To successfully develop complex reasoning skills like these, students need plenty of practice, delivering plenty of feedback

.

When it comes to developing complex reasoning skills, simpler tasks like multiple-choice with right vs. wrong feedback

fall short. They capture only an answer, not the nuanced thought which produced it. That means these types of exercises can’t possibly give detailed feedback on the student’s thought processes: their thinking never got recorded in the first place.