Alpha School and 2x Learning

Alpha School in the “News”

Alpha has been popping up in my corner of the internet lately, mainly because it was a finalist in Astral Codex Ten’s annual Everything-Except-Book Review Contest. Take a look: Your Review: Alpha School.

My two kids attended Alpha until we went abroad last year and, in general, we like what it’s trying to accomplish and agree with the big picture goals. I could write my own entire review that would look rather different than the ACX review above, but instead I’d like to focus on one aspect that’s more thematic to this website: the math behind “2x Learning”.

The Claims

Before we get into the details, here’s the basic thesis of this post: people throw around a lot of claims about “2x Learning” in the review and the comments. I’ve pulled out a handful of the most common ones below (along with my responses) - and I’ll demonstrate exactly why they don’t hold up later in the post.

1. From the Review: “When Alpha says their kids are learning 2.6x faster than kids in traditional schools, what they mean is that Alpha kids are increasing their MAP scores 2.6-times faster than similar kids at traditional schools.”: Alpha kids are increasing their MAP scores faster than kids at traditional schools, but the number is not 2.6 times faster. 2.6 is an erroneously computed and meaningless number.
2. From the Review: “Kids at the GT-school advance ~5x faster.”: 5 is also an erroneously computed and meaningless number.
3. Mark Roulo asks: “I haven’t read the entire review (yet) but 2-3x sustained would have the kids performing at college freshman level by about grade 5. Do they? Or does the 2-3x mean something else?”: No 5th grader at Alpha is at the college freshman level by grade 5. 2-3x is an erroneously computed number.
4. DangerouslyUnstable says: “So top level 3rd graders get the same MAP score (multiple choice testing) as median high schoolers. So they probably don’t understand Newtonian physics or read the Illiad…because median high schoolers can’t do those things either.” No, some 3rd graders are getting a MAP score on the grade 2-5 test that happens to be numerically equivalent to what a median high schooler is getting on a different test that covers completely separate material.
5. Scotty Smyth says: “So my guess would be, yea, a fifth grader in this system would probably be able to read and understand Fagles’ Iliad … .”: No fifth grader in Alpha can read and understand the Iliad.
6. Seta Sojiro says: “The core claim that with this well optimized program students learn 2.6x more in two hours per day than traditional students learn in 8 hours per day. So it’s really a 10x speed up.”: Nobody at Alpha is learning 2.6 faster. They are certainly not learning 10x faster.

It’s worth saying outright: I do believe kids at Alpha probably are learning faster than they would in a typical classroom - at least in some subjects, for some kids. But is it 2.6x faster? No. That number is pure marketing fiction.

So what is the “2x” in “2x Learning”?

The ACX review almost gets the definition right:

When Alpha says their kids are learning 2.6x faster than kids in traditional schools, what they mean is that Alpha kids are increasing their MAP scores 2.6-times faster than similar kids at traditional schools.

What that means in practice is that kids at Alpha improve their percentile ranking on MAP results every time they take the test. If a 3rd-grader at Alpha scores a “209” on Math in the Spring (71st percentile), you can expect she will achieve (on average) a 235 the following spring when she is in 4th grade (traditional 71st-percentile 3rd graders improve ~10 points, so her experience at Alpha should have her improve 10 x 2.6 = 26 points).

Replace the word “average” with “median” and it’s essentially correct. Alpha calculates each student’s growth compared to the median expected growth for their peers, kids across America at the same starting percentile and grade. Then they average all those ratios to get their big “2.6x” number.

Aside from how weird it is to average a bunch of multiplies of medians, the deeper problem is that this approach is statistically meaningless. It ignores how spread out the scores actually are - so the same ratio can mean wildly different things depending on the data’s standard deviation.

It’s tricky to explain why this breaks down using MAP scores directly, because some people reading this know the scale and might anchor on familiar percentiles or test score ranges. To avoid that mental baggage, I’m going to use a weird analogy instead so you, the reader, don’t come in with any preconceived sense of what “normal” looks like.

So here’s an example with a made-up currency you’ve never seen before: gribnobs.

The Gribnob Problem

Imagine a fairy comes around once a year and gives everyone on Earth some gribnobs - little magical tokens everyone wants more of. After dropping them off, she whispers how many you’ll get next year and what the median “raise” will be.

This year, the median gribnob raise is 10. You’re getting 20. That’s double the median. Sounds great, right?

Well, maybe. It depends on the spread. If the standard deviation is 5, you just beat 95% of people - impressive. But if the standard deviation is 100, you’re basically average, sitting at the 54th percentile.

Same “2x median” growth. Totally different reality.

MAP Scores Aren’t Any Different

This is exactly what’s broken about Alpha’s “2x Learning” math. They take a student’s MAP score growth, divide it by the median growth, and call it a multiplier - ignoring the spread of the data.

Let’s say the median MAP growth for a 3rd grader is 10 points. If a student gains 26, that’s “2.6x Learning.” But without looking at the standard deviation, that number is meaningless. If the standard deviation is 10, that’s a big deal - you’re deep in the top percentiles. If it’s 30, you’re barely above average.

Between grades 5–7, this fudge factor isn’t obvious because the median and standard deviation just happen to be numerically similar. So the ratio roughly tracks a real z-score. But at other grade levels, it breaks down spectacularly.

Where It Goes Off the Rails

Outside of the 5-7 grade band, the coincidence disappears and you get some crazy behaviour.

High school is the worst offender. There, median growth in certain places is only 0.85 points, with a standard deviation of approximately 6. If a kid bumps their score by 9 points (about 1.5 standard deviations), Alpha’s formula spits out:

$$9\div0.85 \approx 10.6\text{x Learning}$$

Wait, what??

So a modestly above-average bump magically becomes “10x Learning.” Absurd. And then they average that nonsense across all students to announce a school-wide “2.6x Learning” metric. It’s just bad statistics.

Why Hasn’t Alpha Fixed This?

Short answer: they don’t want to.

I’ve proposed alternatives - better metrics that would actually mean something. They were rejected. Apparently “1.4 Sigma Learning” doesn’t look good on a slide deck. “2x Learning”, on the other hand:

• Is easy to explain to parents.
• It sounds impressive.
• It sells.

I understand and empathize with all of these points. It’s catchy, it’s easy to sell to parents, and it feels good to believe your kid is “2x ahead.” But making up feel-good numbers for marketing shouldn’t be the mission of any school. If you’re serious about education, you owe it to your students - and their parents - to ground your claims in reality.

Truth should be the baseline, not a nice-to-have. If the actual progress is impressive, it can stand on its own. If it’s not, no amount of creative math fixes that.

Of course, it’s not up to me to decide how Alpha spins its story. I love the school, but as long as they keep pushing the “2x Learning” fantasy, someone needs to say it: the emperor has no clothes.