More enrollees meant more test takers, many of whom came from middle- or lower-class areas, where schools and the quality of education typically received less funding and less community support. In other words, many of these students had attended lower-quality elementary and secondary schools, and this affected their test scores. While experts caution that this phenomenon is only one of many factors affecting test scores, they agree that it has contributed to a lower nationwide average. So if scores haven't changed, it must mean kids are smarter now.
Got that? "Same" actually means "smarter."
Oh, but wait. Since 1963, a booming test-prep industry has emerged. Many kids now train for the SAT in classes that promise to raise their scores by as much as 100 points. So, wow, if scores are unchanged since 1963, it must mean students are dumber now.
Got that? "Same" means "dumber."
I could give you four more oh-but-waits just on the SAT, but you get the point. The soil under every statistic is crawling with worms.
One more example--I can't resist:
A massive study done in 1970 showed that 25 million adult Americans were illiterate. A similar study released in 1992 put the number closer to 85 million. Now can we panic?
No, not yet. Look closer.
From 1970 to 1992 the definition of "illiterate" changed. In 1975, if you could sound out the words "bus schedule," you were literate. Today, if you can read every word in the bus schedule but can't use it to catch a particular bus, you're illiterate. I'm not saying the new definition is wrong. I'm just saying you can't tell--from these numbers--if illiteracy has gone up or down.
Let's consult some experts
I called Tom Williamson, a former president of the Psychological Corporation (one of the big three of American test publishing). "Do you think kids are getting dumber?"
"No." His answer was so emphatic and immediate--it almost preceded my question. "We always tend to complain about the achievements of the current generation and exaggerate the accomplishments of our own."
True of me, certainly. Why, when I was a kid ...
"I think both schools and kids are doing a better job than they ever have," Williamson said. "You have to take into account that classrooms are much more diverse now. With mainstreaming, you've got kids with physical and emotional problems in regular classrooms. Students who used to be excused from taking standardized achievement tests are no longer excused. If you test a broader range of kids you're going to get a slightly lower score."
Williamson then brought up the "mystique of testing" question, which is: Do standardized tests really show whether kids are getting dumber?
"We test what's easy to measure, not necessarily what's important," said Williamson.
He'd get no argument from the Educational Testing Service, the people who create the SAT. I talked to Tom Ewing, communications director of ETS. What I got from him was a carefully crafted statement that sounded like boilerplate created to beat back the millions of reporters who call in every day to ask, "Are American kids getting dumber?"
His bottom line: The SAT scores can't tell you.
"Why not?"
"Because," he said, "the sample is self-selected."
In other words, students themselves decide who among them will take the SAT. There are no controls. Here's an example to illustrate the point. Suppose you go to a mall and weigh everyone who lets you. Then a month later you go back to the same mall and weigh everyone who lets you. If the numbers are higher the second time, you can't conclude that people are getting fatter. All you can conclude is that more heavy people participated the second time around.