Man—I had a whole, scathing essay written and ready to go.
The title: The SAT Changed Their Guessing Policy to Appear Fairer, But It’s Actually Less Fair. “With the ACT pulling ahead in the admissions test Cola Wars,” I wrote, “I struggle to greet the SAT’s announced changes with anything but cynicism.”
I was halfway into the boxing ring when I realized I was on the wrong side of the fight.
This little fable is about the SAT’s “guessing penalty,” and while it’s a tale full of technicalities, I promise it’ll end with a moral. A moral so obvious, it’s surprising.
Or perhaps vice versa: so surprising, it’s obvious.
Although the term “guessing penalty” appears practically everywhere (including the New York Times), the College Board never uses it. And the College Board is right. There’s no way to penalize guessing, per se—after all, the SAT only sees right answers and wrong ones. They’ve got no way of knowing whether you arrived at your response by cold, logical deduction or by blind, stupid luck.
Instead, the current SAT penalizes wrong answers. You get 1 raw point for answering a question right, and lose ¼ of a point for answering it wrong. (A blank answer neither adds nor subtracts from your total.)
This system doesn’t aim to penalize guessing. It aims to neutralize it.
Let’s say you guess blindly on five questions. On average, you’ll get one right (+1) and the other four wrong (-¼ – ¼ – ¼ – ¼), for a net score of zero. Since the outcome is no different than if you’d simply left all five questions blank, guessing should neither help nor hurt you overall. It just adds a greater element of randomness to your score.
Why did the College Board ditch this longstanding system? For simplicity’s sake.
In their announcement, they described the old system as “Complex Scoring.” That’s fair. For their whole lives, students have taken tests with no distinction between blank answers and wrong answers—both are worth zero. Then, on the SAT, when the stakes are the highest, the rules of the game suddenly switch, and blank becomes (slightly) better than wrong.
This unfamiliar system might seem to confer an advantage on wealthy families. After all, they can afford tutors to explain the rule and its implications, while poorer students remain in the dark.
Hence, the new and cleaner system: One point for right answers. Period.
Unfortunately, this revision doesn’t make the test less gameable. It makes the test appear less gameable, while actually making it more so.
To see why, suppose that you and I both take a timed section with 20 questions. Being similarly skilled (and rather slow) students, we each get through question #10, finishing it with a few seconds to spare.
But then, I spend those final few moments working (naturally, it would seem) on problem #11. You, on the other hand, spend the remaining time randomly bubbling answers for #11-20.
Clever move on your part. Under the old rule, this would’ve merely added randomness to your score. But under the new rule, you’ll get credit for your right guesses and suffer no cost for your wrong ones. On average, it’ll boost your score by 2 points—a 20% improvement over mine.
You and I knew precisely the same number of actual answers, but you did significantly better, because you employed the right test-taking strategy.
Isn’t that precisely the scenario the new SAT is supposed to help avoid: rewarding students not for their knowledge of the content, but of the test?
But this is where I got it wrong, and where the SAT—to their credit—got it right.
The crucial fact is that, even though it’s perfectly possible to earn a solid score (say, a 600 per section) while answering only two-thirds of the questions, virtually no students take that approach. Out of every 25 questions, the typical student answers 24.
It’s strange—the system gives them no incentive to do this. They’d largely be better off focusing on 12 or 14 questions, making sure to get them right, rather than spreading themselves thin by hurrying to finish. Even so, students answer virtually every question, leaving almost no blanks.
The SAT’s revision, then, is just a response to the facts on the ground. Kids are already answering all the questions. In this landscape, penalizing wrong answers, on top of rewarding right ones, is redundant and confusing.
And here we come to the moral, the take-home message, the thing I learned from picking a losing rhetorical battle with the College Board, and then wising up.
It’s vacuous to call a test “good in theory.” For an assessment, the only thing that matters is practice. The measure of a test’s success lies in its interactions with the humans that take it.
When designing (or pondering the design of) a behemoth test like the SAT, it’s tempting to think of it as a cold, impersonal system of incentives. It’s easy to forget that it will land on the desk not of some abstract test-taking machine but of real, flesh-and-blood students.
As such, it must obey the imperative of every test: to respond to the needs and habits of the test-takers.
A test may appear transparent, fair, and perfectly organized to an outsider, but if it confuses the students and obscures their abilities, then it’s worthless. An ingenious bit of test-craft gets you nowhere if it asks students to fight against instincts that they simply can’t overcome.
So College Board, I apologize for the hate-letter I never sent. You were right.