Tutoring Science

The 2 Sigma Claim

In 1984, Benjamin Bloom reported that students tutored one-on-one with mastery learning performed about two standard deviations above students taught in conventional classrooms.

The finding became known as the 2 sigma problem.

The figure is widely cited. Later work suggests it overstates typical effects.

Large syntheses of randomized K-12 tutoring trials still find substantial positive effects. The strongest results come from frequent, high-dosage tutoring built into the school day.

Granularity Is the Lever

So what makes tutoring work? Kurt VanLehn's 2011 review compared human and computer tutoring and pointed to one factor: granularity of interaction.

Granularity means how small the unit of interaction is.

Some tutors only check final answers. Others interact at the level of individual problem steps.

Step-level tutors can detect errors early and see where the reasoning broke.

This reframes the question. The one-on-one format is the setting; the mechanisms inside it do the work.

The Active Ingredients

Step-level interaction enables three core mechanisms. Treat them as your checklist.

Frequent, step-by-step interaction. The student works and the tutor responds often.

In-the-moment diagnosis. The tutor identifies the misconception behind an error.

Immediate, adaptive feedback. The response fits this student's actual mistake.

The literature proposes more: prompting students to explain their reasoning, scaffolding that fades as competence grows, and motivational support from a relationship with the tutor.

A Student Writes 2/7

Imagine a 6th grader working on $$\frac{1}{3} + \frac{1}{4}$$ who writes $$\frac{2}{7}$$.

A human tutor and an AI math tutor each have to decide how to respond.

One option is to mark the final answer wrong. That is quick, but it says nothing about why 2/7 came out.

The other option responds to the thinking behind the answer. That choice decides whether the misconception gets fixed or quietly reinforced.

Fixing It Step by Step

Suppose the tutor responds at the level of steps.

Diagnose. Getting 2/7 means the student added numerators and denominators separately, a known misconception.

Respond to the cause. Ask whether $$\frac{1}{4} + \frac{1}{4}$$ could equal $$\frac{2}{8}$$, which the student knows is the same as 1/4.

Give immediate feedback as the student rewrites with twelfths: $$\frac{4}{12} + \frac{3}{12} = \frac{7}{12}$$.

The misconception is corrected. An AI tutor would need these same three steps to count as equivalent.

Judging an AI Tutor

In VanLehn's review, step-based intelligent tutoring systems produced effects close to those of human tutors. Matching the mechanisms is possible.

So talking with a student one-on-one is not the test. Reproducing the mechanisms is.

Ask: Does it diagnose errors step by step? Does it adapt its feedback? Does it keep students actively reasoning instead of handing them answers?

Then ask whether it sustains enough engagement to deliver a meaningful amount of practice. The strongest human results came from high-dosage tutoring.

Bloom's 1984 finding that tutored students scored about two standard deviations above classroom students is best treated as _____ of typical tutoring effects

• an overstatement
• an accurate estimate

Answer: an overstatement

Bloom's '2 sigma' figure is widely cited, but later work suggests it overstates typical effects. Large syntheses of randomized K-12 trials still find substantial positive effects, just smaller ones. For product claims and research planning, the 2 sigma figure is a ceiling, not a benchmark to promise.

Beyond step-level feedback, the tutoring literature proposes other ingredients: _____ that fades as competence grows, _____ prompts that ask students to state their reasoning, and _____ support that comes from a relationship with the tutor.

Answer: Beyond step-level feedback, the tutoring literature proposes other ingredients: scaffolding that fades as competence grows, self-explanation prompts that ask students to state their reasoning, and motivational support that comes from a relationship with the tutor.

Scaffolding is temporary support that should fade as the student improves. Self-explanation prompts keep students actively reasoning. Motivational support comes from the tutor relationship and helps sustain engagement. Each one is a mechanism an AI tutor can be checked against.

Kurt VanLehn's 2011 review compared human tutoring with computer tutoring. What did it argue mainly explains tutoring's benefit?

• Interacting at the level of individual problem steps
• Having one adult assigned to each student
• Building a personal relationship with the tutor
• Spending more total minutes on each topic

Answer: Interacting at the level of individual problem steps

VanLehn argued the benefit depends largely on the granularity of interaction. When a tutor responds to individual problem steps, it can catch errors early, diagnose the misconception, and give immediate, adaptive feedback. Step-based intelligent tutoring systems produced effects close to human tutors. That result suggests the mechanism, not the human or the one-to-one ratio, does most of the work.