Success can’t be measured by one or two numbers

“Whenever you make huge decisions about complex situations based on one or two numbers, you’re headed for disaster — especially when those numbers can be gamed.”

—Mark Palko and Andrew Gelman, “How schools that obsess about standardized tests ruin them as measures of success” on Vox

We’ve questioned Success Academy’s “success” on this blog before. These statisticians bring a new lens to that question.

I don’t want to denigrate the good work that Success Academy teachers and students are doing. There are practices and systems well worth replicating and investigating in these schools. But Eva Moskowitz’s political framing and marketing of her schools as the solution to poverty is problematic.

What Is Success?

Ecoschool

In an interesting coincidence, Will and I have been writing about Eva Moskowitz and poverty, and here the NY state test results come along. Success Academy made up 7 out 15 of the top scorers in NY state, according to the NY Post. And while of course poverty level correlated with reading performance, there were the outliers of high poverty schools that outperformed state averages.

But Success Academy was more than just a freak outlier—it blew other schools out of the water, and it did so consistently across multiple schools.

In the NY Daily News, Robert Pondiscio writes in “How Does She Do It?“:

What is imperative now is for serious, unbiased experts and observers to descend on Harlem and figure out how these extraordinary results are being achieved and, if all that glitters is gold, how to replicate them.

It’s worth repeating a challenge I threw out earlier to Eva Moskowitz in a recent post: share the practices, content, and protocols your schools are using so others can benefit. That’s what being a “public” school is all about, right? Collaboration, sharing, learning. That’s what our public system of education should be doing if we’re truly dedicated to improving outcomes for all of our children, and not just some.

As John King also said: “The question becomes, what’s happening in these schools that’s leading to those better outcomes?”

I agree with Pondiscio and King. I genuinely want to know what’s happening at Success Academy. And I want to know from the perspective of a school as an ecosystem. From this perspective, some questions I would ask would be oriented around the main pillars of a strong school community: 1) leadership, 2) content, and 3) environment.

Leadership

  • What do the formal and informal leaders say and do? How and what do they communicate consistently? (This includes student leaders).
  • Is the leadership distributed?
  • What mechanisms are in place for students, parents, teachers, and leaders to collaborate and receive continuous feedback? How do leaders respond to feedback?
  • How is diversity in student ability, knowledge, and skills strategically recognized and cultivated?

Content

  • What are the values and vision behind assessment and unit design?
  • What texts are taught in ELA? Why?
  • How well do topics and themes build knowledge and understanding of academic domains and the world sequentially across classrooms and grades?
  • How are students engaged in their community through units?
  • What scaffolds and interventions for students who are struggling are applied consistently both in and out of classrooms?
  • What opportunities beyond academics are provided for all students?

Environment

  • What does it feel like when you walk into a Success Academy school? What does it sound like? What does it look like?
  • How relevant is posted work and displays to students and their community?
  • What is the ratio of positive to negative language used by students and staff in the building?
  • How (psychologically) safe do students with special needs feel in the hallways, lunch rooms, and classrooms?
  • How are supportive social relationships and networks developed and sustained by the school?

These are just some of the questions I would start with.

State test results are just one indicator. Remember, we’re playing the long game here. And there is evidence to suggest that test results are a shallow indicator.

I don’t say this to put down what Success Academy has accomplished. Their students deserve our recognition. They’ve worked hard for this.

But what I want to know is whether what Success Academy is doing is truly preparing students for the future, for the long-term.

What is success? And what does it look like? And is what Success Academy doing truly an exemplar?

And more importantly—what does success look like over the long haul?

The Myth of the School as Savior of All Social Ills

Koti Chennayya – Legendary twin heroes known for their martial skills, heroic deeds, discipline and above all strength of character and respect for truth.

Eva Moskowitz recently wrote an op-ed for NY Post in which she makes the case that schools can effectively overcome all obstacles. In my last post here, “An Editorial on Societal Culpability for Have-Nots,” I argued something somewhat different. So it would be worth examining her argument closer so I can clarify my position.

Eva starts off with a strong statement:

There is a myth in this country that poverty and race are overwhelming barriers to a child’s ability to learn. This is simply not the case.

She then promotes the exemplar of her own schools, the Success Academy charter network, as the supporting evidence for this statement. “Success Academies are free, K-12 public schools, open to all children. . . . Success Academy schools are at the top of all public schools in the state.”

This is a bit of an aside, but I’d like to throw down the gauntlet here for Ms. Moskowitz. If she is truly committed to the “public” part of education, then why not share all of the wonderful practices and content that makes her schools so successful? I’m someone always seeking to learn from best practices, and I work willingly across charter and district divides.

When the scores came out for the first Common Core-aligned NY state tests last year, and I saw Success Academy II in the Bronx had the highest scores in the city, I tweeted that I wanted to know what they were doing. I’m serious about that.

Share, Ms. Moskowitz, not simply compete. That’s how we can make all our schools better, and not just yours.

Moskowitz then makes the following statements, in which I can find much to agree with:

If we sell low-income, minority children short, because we believe their poverty prevents them from learning, then indeed, they won’t learn. If we want to help our children of color to rise out of poverty, we must give them schools on par with what their more affluent peers have. . . .

I fully concur that we need to make our schools better for students of color and students in poverty AND for students with special needs, and that we must raise our expectations for ALL children. This is why I teach and this is why I blog.

This also made me think about a recent post on Shanker Blog by William Schmidt, “The Global Relationship Between Classroom Content And Unequal Educational Outcomes,” in which he says:

It is very unlikely that students will learn material they are not exposed to, and there is considerable evidence that disadvantaged students are systematically tracked into classrooms with weaker content. Rather than mitigating the effects of poverty, many American schools are exacerbating them.

We need to make our schools better. Our schools are not good enough, particularly for our students who rely on them for opportunities the most.

Moskowitz closes her op-ed with the following statement:

We don’t need so much to “lift” children from poverty as to equip them with the skills and self-confidence to achieve their dreams. We must choose to make schools incubators of opportunity, not poverty traps.

Though I agree with her sentiment, I also think this kind of wishful thinking can be problematic. You can see more evidence in the following statements that Moskowitz made earlier in the post:

We as a nation can’t fix poverty unless we fix education, and we can’t fix education if we keep telling ourselves our schools are “good enough.” . . .

If we give all children a fair start, then the race is theirs to win.

Why do I call this “wishful thinking”? Because essentially, Moskowitz is arguing that if we fix everything in the classroom, then we’ve provided our students of poverty and color with ample opportunity and equity. Then it’s off to the races. Our wonderful American meritocracy will then function as it should.

But it won’t. It won’t because even if we provide the best education in the world in an isolated pocket of poverty, what opportunities will the children in that community have upon their graduation? What social networks will they have to support them as they climb to ever more challenging and higher rungs in professional and academic settings?

Fixing education is not enough, and saying that it is enough is in my opinion wishful thinking because it lets us off the hook, and this letting-us-off-the-hook is why I wrote my last impassioned post about our society’s culpability for the horrendous living situations and life outcomes of too many of our nation’s children. If we think we can fix everything in the classroom, that we can dust off our hands at the end of the day, get into our BMWs and drive off to our wealthier enclaves, eat our organically grown fresh produce, and tell ourselves that we’ve done everything we could, then we are fooling ourselves, and we are letting ourselves and those in power in our society off the hook.

Because making schools better also isn’t “good enough.” It’s important, and it’s the battle Moskowitz and myself and educators throughout our nation get up each morning to do, because we believe in its importance. But if it’s good enough, then we are effectively saying that we are endorsing socio-economic and racial segregation. We are effectively saying that so long as you have yours, and I have mine—and it’s equal—then everything’s going to be OK.

It’s not going to be OK, so long as there’s the other side of the railroad tracks. And it’s not going to be OK when even if we provide a world class education such as Success Academy is claiming to do, our students living in poverty still are not completing college at greater rates nor obtaining higher paid careers (is Success Academy tracking longer term outcomes? That’s what KIPP is doing–they’re taking a hard and honest look at the graduation rates of their scholars, rather than giving themselves high fives).

I love Moskowitz’ idea of schools as “incubators of opportunity.” I’d like to extend that idea to entire communities. The communities of the South Bronx and Brownsville must be cultivated as incubators of opportunity, not just their schools. Detroit and East LA and South Texas should be nurtured and invested in as incubators of opportunity.

A school is part of an ecosystem of a community. What economic opportunities are available? What social and physical capital investments have been made? What sort of public transportation options are there?

Even if we had the best schools in the world in our poorest communities, our work has only just begun. Education is important. But that’s only half the battle.