Good Data Instead of Bad

I'm finally reading Michael Lewis's book Moneyball and the story of a writer named Bill James who complains about the error statistic in baseball.

It is, without exception, the only major statistic in sports which is a record of what and observer thinks _should have been accomplished_. It's a moral judgment, really, in the peculiar quasi-morality of the locker room. . . . But the fact of a baseball error is that _no_ play has been made but that the scorer thinks it should have. It is, uniquely, _a record of opinions." Lewis goes on to say that "The statistics were not merely inadequate; they lied. And the lies they told led the people who ran major league baseball teams to misjudge their players, and mismanage their games.

Well, doesn't that sound just like high stakes testing in schools?

The tests measure student error. They are designed to see who doesn't know the answer because the hypothesis is that US students aren't learning well. Kids have to get a lot wrong on the tests in order to justify education reform. The tests aren't measuring achievement so much as counting errors and making moral judgments as to what should have been accomplished so as to justify an entirely new game plan.

The tests then measure teacher errors. Kids got questions 18, 23, and 79 wrong, so those things aren't being taught well. Kids have to cleanly field these questions or their teachers must be send to the minors or cut from the team.

Schools that fail to reduce errors are teams that have to be disbanded, new coaches brought in, all players traded away, and a new draft taken.

Here's the thing that has to be kept in mind: the errors are subjective calls on tests meant to show errors. It's bad data being used badly and in a big way. 

I'm curious about other data.

I would start by asking kids what their best class is and why. In what class do you feel you are learning and why? Parents, consider the subjects in which your kids do best and those with which they almost always struggle. Teachers, pick the colleagues you most respect and say what is so admirable about them. And I would ask students, four years after the fact, which teachers/classes they remember and why.

I want schools to collect data similar to slugging  and on-base percentage. In other words, I want data associated with learning more than testing.

I'd like a better poverty measure than how many kids are on reduced or free lunch.

I want to measure school happiness and compare that to attendance. If a kid comes to school, that kid has a better chance of learning what the school teaches.

I need to get into this more and I want to talk with teachers, parents, administrators and education professionals at Syracuse University. I want to see this kind of research. I want data to make positive change instead of punitive change.

Bad data isn't any good. Most test data is bad. We need to ask the right questions, keep the right records, and find a way forward.

Write on.