Social planners must feel the weight of history

When Nick Kristof recently extolled Oklahoma's universal pre-school for four-year-olds, it was deja vu for those of us who are hard-headed about evaluating social programs.

Oklahoma pre-school has never been subject to a "gold standard" evaluation--meaning a large-scale, randomized experiment. Kristof's column came just weeks after a gold standard evaluation of pre-school in Tennessee found essentially no differences between the treatment and control groups. And that evaluation follows decades of experiments showing little to no effect of similar programs. Nevertheless, Kristof's enthusiasm for Oklahoma is unchecked by the weight of history.

There's a pattern here. As I wrote for National Review recently:

Skeptics of government social programs have long been frustrated by a familiar pattern of media coverage. It starts with a new social program portrayed as the key to fixing problems like poverty or low test scores. The media proceed to either ignore the long history of similar initiatives found to have little effect, or they acknowledge the history but report that this program is fundamentally different from everything that has come before.

Either way, the media trumpet the positive results from early evaluations of the program, even when those results are based on small samples, short time horizons, non-experimental data, or all of the above. Years later, researchers find that a randomized experiment shows that the program has no effect, or that the effects quickly fade, or that the effects cannot be replicated on a larger scale. These findings are quietly reported in a technical article, with no fanfare and little public attention. By that time, the media have already moved on to promoting another new social program.

Read the whole thing here.