Jim Palermo: Into the standardized test looking glass

Last modified: Friday, January 08, 2016

It took 18 months of sustained effort, and a hearing in Hampshire Superior Court, to compel the Department of Elementary and Secondary Education (DESE) to release what turned out to be a one-page typewritten list of the of 49 Massachusetts schools and the 47 school districts that participated in the 2012 round of student testing under the Program for International Student Assessment (PISA).

It was this test that ranked Massachusetts students among the best in the world.

I am grateful for the support I received from the Daily Hampshire Gazette, the free legal services provided by attorney Thomas A. Miranda and the expert affidavit testimony of Dr. Arnold D. Well, professor emeritus at UMass Amherst. I am also grateful to state Rep. Peter Kocot and Northwestern District Attorney David Sullivan for their guidance and moral support.

While they may not share my views regarding education reform, each of them believed in my right to pursue this matter under the state’s Public Records Law.

When the list of schools arrived, I began to analyze it to determine whether the identified schools — none of which were charters — were representative of all the schools in Massachusetts. I compared the median incomes of their respective districts with the median incomes of districts that did not participate in PISA and attempted to determine how they compared to other schools with regard to MCAS and PARCC testing.

I then attempted to write an article about what I learned, only to scrap draft after draft, because I became bogged down — overwhelmed, in fact — in statistical minutiae. Then it dawned on me: I had allowed myself to be drawn into the game. The fact is that the obscene amount of statistical analysis and data that has been generated as part of the standardized testing/school reform movement can be manipulated to prove whatever one wants it to prove and is, therefore, largely meaningless.

The statistic that is most significant is how well the students in a particular school are learning the material they are being taught.

If classroom testing reveals that some students have not grasped working with fractions, the teacher is put on notice that intervention is needed. At the classroom level, that intervention might include more than traditional tutoring: it might also include being able to accommodate a child who finds it difficult to concentrate because of circumstances outside the classroom — a sick parent, insufficient sleep, the death of a pet, or stress, for example.

That is the type of concern we would expect from our doctor — treatment for the specific factors affecting our health, not a course of treatment based on statistics regarding the population at large. Think of a doctor prescribing a diet and exercise to a thin patient, based on statistics showing that too many Americans are overweight.

As silly as that sounds, that is exactly where standardized testing is leading us. And I suggest that we are being led in that direction not because researchers are stupid, but because that is where money is to be made by those selling education-technology, marketing standardized tests and seeking to privatize public schools — an industry that Rupert Murdoch claims is a $500 billion market in the United States alone.

A convincing argument, using actual statistics, can be made that traditional public schools greatly outperform charter schools, based on 2013 MCAS scores for math, among 10th graders. The top five rated charter schools had a total of 437 students (96.9 percent) performing at advanced and proficient levels. But the top five ranked traditional public schools had 1,396 students (99.2 percent) at the advanced-proficient level: more students with better results.

The only problem is, such statistics — as much as they support my pro-public school bias — tell us nothing about why some schools do better than others and ignore the myriad variables that factor into human behavior.

Thus, the statistics I cited above — while appearing to be convincing evidence in support of my position — are actually bad statistics, just like so much of the data that drives our “race to the top” in education. Should it surprise anyone, for example, that 99 percent of the 10th grade students at the Pioneer Valley Performing Arts Charter Public School attained rankings of advanced or proficient in English on the 2013 round of MCAS?

I suggest a major reason for their excellent results is that students who love theater are students motivated to love language — a qualitative variable unaccounted for in any statistical analysis of MCAS results.

I am convinced that meaningful education reform can be attained only if we focus on the education of each individual child. I suspect, however, that in most instances what is keeping children from success has more to do with poverty and complex societal factors which politicians and their rich contributors are unwilling to confront.

Jim Palermo lives in Southampton.


Daily Hampshire Gazette Office

115 Conz Street
Northampton, MA 01061


Copyright © 2020 by H.S. Gere & Sons, Inc.
Terms & Conditions - Privacy Policy