VAM and School Ecosystems

Wiring diagram for “Wilson’s Central Pattern Generator” By Thomas.haslwanter (Own work) [CC BY-SA 3.0]
An excellent essay published in March’s Educational Researcher (1) by Susan Moore Johnson (2) brings the lens of complex systems to bear on the controversial issue of VAM.

Johnson takes to task the hasty moves of policymakers to bring VAM to the forefront of teacher evaluation. She focuses specifically on the problematic assertion by proponents that VAM is an effective measure to isolate and measure individual teachers. Johnson brings what she terms an “organizational perspective” to the issue—in other words, she considers how VAM might play out from the perspective of a school as an ecosystem, rather than as a set of individual, isolated teachers.

Johnson’s perspective is unique in that she explicitly acknowledges the impact of social networks on teacher and student performance. She refers to a 1988 James Coleman analysis, “Social Capital in the Creation of Human Capital,” which suggests that:

. . . whatever human capital schools acquire through hiring can subsequently be developed by interactions among teachers, principals, and others within the organization through activities within subunits such as grade-level or subject-based teams of teachers, faculty committees, professional development, coaching, evaluation, and informal interactions. In the process, the school organization becomes greater than the sum of its parts, and in this way, the social capital that transforms human capital through collegial activities in schools increases the school’s overall instructional capacity and, arguably, its success.

It probably goes without saying that the recognition of a school organization as “greater than the sum of its parts” is something we’re pretty adamant about here on Schools & Ecosystems.

Johnson asserts that the traditional structure of a school as an “egg crate,” in which teachers are isolated from one another, is a problematic model:

Compartmentalized school structures limit the potential development of individual teachers, who lack direct access to their colleagues’ expertise. However, social capital theory would suggest that if provided systematic opportunities to engage with their peers outside their classroom, the human capital of individuals—in this case, their instructional effectiveness—could be shared and augmented. Given this line of argument, the more robust the teachers’ instructional repertoire and the more opportunities they have to exchange and integrate promising ideas and techniques into their own teaching, the more likely it will be that all students—not only those assigned to the more effective teachers—will experience the benefits of expert teaching. This analysis suggests that teachers are not inherently effective or ineffective but that their development may be stunted when they work alone, without the benefit of ongoing collegial influence.

I’ve personally worked as a teacher in two very different kinds of school environments: one in which I was mostly isolated from my peers (due to divisive internal politics and management), to one in which I meet daily with colleagues to discuss student work and instructional practices. And I can attest to the significant impact that a collaborative and positive organizational environment has had upon my professional development.

Johnson would phrase my experience as an example of “how social capital augments human capital.” She points to the influence of context and peer learning as a factor in professional development:

. . . changing the context in which teachers work could have important benefits for students throughout the school, whereas changing individual teachers without changing the context might not (Lohr, 2012). Given that possibility, it is worth learning more about the components of a teacher’s workplace that promote greater satisfaction and more interdependent work.

. . . both theory and empirical evidence suggest that students and their schools stand to benefit when teachers work closely and collaboratively with colleagues.

Johnson also explores the “unintended consequences” of relying on VAM to evaluate individual teachers, a warning that we’ve echoed here in response to Gov. Cuomo’s misguided push for greater weight on student test scores, and which also echoes similar issues that Dana Goldstein has pointed to in her historical exploration of teacher evaluation.


She concludes that “that expanding the use of VAMS in teacher evaluations (even if it represents no more than 30% of the teacher’s total score) might compromise the school’s potential for improvement.”

Johnson therefore suggests that “reformers should lead the way with efforts to improve the school throughout as an organization that supports effective teaching and rich learning.”  She suggests including teachers in the hiring and professional development process in their schools, as well as in recognizing and sharing instructional practices from teacher leaders, such as Peer Assisted Review (PAR) programs.

This is the kind of thinking that I wish we heard more of in education reform circles—not hasty, breathless championing of the latest panacea, but instead reflective approaches that consider schools as whole organizations, complex adaptive systems ripe with social interactions and relationships.

1 Educational Researcher, Vol. 44 No. 2, pp. 117–126 DOI: 10.3102/0013189X15573351 © 2015 AERA.

2 I was alerted to Susan Moore Johnson’s essay by a link to a Marshall Memo from an April 9th USDOE Teacher’s Edition newsletter.

The Deficit of Wisdom On the Common Core


I’ve publicly made the case for Common Core standards on a number of forums (The Core Knowledge Blog, Chalkbeat, VIVA Teachers, & Impatient Optimists). I view the standards as an opportunity to align better practices and stronger content, though how the standards ultimately play out is dependent on how we elect to interpret and implement them. Unfortunately, most leaders are opting to play political theater, and too many teachers and parents use the standards as a proxy for problematic state policies or district decision-making.

I’ve since ceased writing about the standards, mainly because I’m too darn busy planning curriculum and developing IEPs (using the Common Core standards as a guide)—but furthermore because the constant barrage of feigned outrage at the standards has caused me to tune out.

But this morning while on the way to work, I read this piece, “The Wisdom Deficit” by Michael Godsey on The Atlantic that seriously got to me, to the point that I feel compelled to write a rebuttal. Here’s why this piece upsets me:

  • It’s written by an experienced ELA teacher who has his heart in the right place: he cares deeply about engaging his students in the wisdom gleaned through the study of classic literature.
  • It points to real problems in how the Common Core are being misinterpreted at the ground level.

After framing the “so-called ‘College and Career Readiness Anchor Standards’ in a dismissively perfunctory manner, rather than analyzing the standards themselves further for evidence of the problems he outlines, he instead points to an “adjunct faculty member in Secondary Education at San Francisco State University” as an appointed spokesperson for the Common Core:

Kate Kinsella, an influential author who consults school districts across the country and is considered “a guiding force on the National Advisory Board for the Consortium on Reading Excellence,” recently told me to “ditch literature” since “literary fiction is not critical to college success.” Kinsella continued, “What’s represented by the standards is the need to analyze texts rather than respond to literature.

As a teacher working within this regimented environment, my classroom objectives have had to shift.

Wait a second. I’ve never heard of Kinsella. She may be influential in California, perhaps, though even that’s questionable given the size and political dynamics of my mother state.  Her “advice” is ill founded, nor supported by a closer examination of the standards themselves. I’ve addressed this before in my post on The Core Knowledge Blog in 2013:

Under key design considerations in the introduction to the literacy standards, Common Core’s authors state that the inclusion of social studies, science, and technical subjects “reflects the unique, time-honored place of ELA teachers in developing students’ literacy skills while at the same time recognizing that teachers in other areas must have a role in this development as well” (bold added).

They furthermore point out that “because the ELA classroom must focus on literature (stories, drama, and poetry) as well as literary nonfiction, a great deal of informational reading in grades 6–12 must take place in other classes” (bold added).

Yet within schools, these points are all too easily ignored or misconstrued. ELA teachers are evaluated by the literacy tests that their students are required to take. One of the greatest frustrations of being an ELA teacher, in fact, is that we are tested on factors that are often beyond our control, such as our students’ domain-specific knowledge. It’s no wonder, then, that many ELA teachers resort to skills-based teaching, grimly attempting to boost test scores by bolstering superficial, isolated skills.

Another thing that really gets me, though: what in the world does Kinsella have to do with “this regimented environment” that Godsey refers to, and why in the world has his “classroom objectives … had to shift”? There’s something seriously wrong in Cali if this kind of misunderstanding is translating into such perceptions of edicts from on high.

Godsey then goes on to suggest that this shift is tantamount to a loss of values (“I get it: My job is to teach communication, not values”), and that as a result, he no longer is able to teach wisdom:

When I recently shared a poem that included the phrase, “Let there be light,” hardly any of my students, who are high-school juniors, could identify the allusion. As a staunch believer in the separation of church and state, I don’t feel comfortable delving into the Bible’s wisdom.

Here’s the irony: the Common Core actually makes it fairly clear that students will require knowledge of the Bible in order to determine allusions made to it. Here’s Reading Literature Standard 9, Grade 9-10:

“Analyze how an author draws on and transforms source material in a specific work (e.g., how Shakespeare treats a theme or topic from Ovid or the Bible or how a later author draws on a play by Shakespeare).” 

Arguably, therefore, Godsey is not heeding the advice of the standards because he is not “comfortable” teaching his students to understand that allusion.

He later states that “there is a noticeable deprioritization of literature, and a crumbling consensus regarding the nation’s idea of classic literature.” Yes, there is. Yet though the Common Core only explicitly references Shakespeare and Ovid in the standards themselves, the standards have made a push for “authentic” and “complex” texts, a dramatic shift from what had been generally prevalent in schools driven by a “test prep” or “leveled book” mentality, in which bland passages were provided to students based on their independent reading levels.

The Common Core standards are not to blame for our nation’s problem with classic literature. Can you imagine if the standards had in any way attempted to dictate what type of texts students needed to read? Yeah. And even as carefully crafted as the standards have been to be politically neutral, they have become so volatile that they are said to threaten Jeb Bush’s candidancy for president (which I think is preposterous—if he sticks to his guns he can reach independents and center-left liberals willing to cross over, but whatever).

Godsey then delivers what I consider the most disturbing part of his piece:

I remember when, 10 years ago, my students spent an hour sharing their favorite lines from Father Zossima’s sermon in The Brothers Karamozov and how and why it affected their own lives. One student was visibly moved by the idea that suffering for a loved one might be a blessing available only in a life on Earth, not in heaven. A few different students called it “their favorite class ever.” This morning, my student-teacher—a college student I’m training to be a classroom educator—used a hip-hop poem as a primary text and started the class by saying, “Today we’re going to practice Reading Standards 1, 2, and particularly 4” in reference to the anchor standards that the students had on their desks. If this sounds a little dry, I’m partly to blame—for a month, he’s been watching me ask the students to explicitly reflect on their progress in each of these technical areas. In any case, with habits like these, he’s sure to land a permanent job in the fall.

Who decided to ask students to “practice Reading Standards 1, 2, and particularly 4” and provided the “anchor standards that the students had on their desks”? Big brother?

At least Godsey acknowledges some of his culpability (“I’m partly to blame”), but he attempts to redirect it towards blame of some obscure “culture” that he just can’t seem to resist paying heed to.

Being transparent with our students about goals and how they are getting evaluated is fundamental, yet asking them to parrot anchor standards strikes me as a colossal waste of instructional time. It’s up to us—the educators—to interpret and apply the standards in our classrooms in a manner that aligns with our knowledge and expertise and with the intent of the standards themselves.

The Common Core are not responsible for poor instructional decisions nor for poor educational consultants nor for poor administrators. The Common Core are an opportunity for educators to fight for better curriculum for all of our children. Let’s stop blaming the standards, and start using them to better teach the values and wisdom and literature so fundamental to our commonweal.

Why Democracy Solves Social Problems Better Than Markets or Hierarchies

The other day I discovered a fascinating essay entitled Cognitive Democracy on the Three-Toed Sloth, thanks to a link on Twitter from The Browser*. Co-written by Henry Farrell and Cosma Rohilla Shalizi (love that name!), it’s a thought provoking examination of the “macro-institutions” of markets, hierarchies, and democracies. As I read through it over Memorial Day weekend, it struck some nerves in me that resonated with related thoughts I’ve pursued here on Schools as Ecosystems and elsewere as I’ve begun branching out in my philosophy regarding education reform.

One thing I’ve struggled with as I mature in my ed reform perspectives hinges on a basic question: what sort of governance structures and decision-making processes are most conducive to the creation of positive and supportive school environments?

I am much enamoured with the need for transparency and “open source” development processes, especially in curriculum development, as I’ve articulated on GothamSchools. I have also written on Education Gadfly about the need for more dialogue and collaboration across all levels of those involved in public education. Central to these ideas is not only the necessity for dialogue, but more fundamentally for a diversity of perspectives to be included within that dialogue, which Will and I have argued for on this blog.

Henry and Cosma hit on some of these points. Ultimately, they argue for the supremacy of democracy over markets and hierarchies as a macro-institution for tackling complex social problems. It’s truly an interesting argument, and well worth spending the length of time it will take you to comb through it. In fact, once you’ve read it, I’d recommend heading over to Crooked Timber, where it is cross-posted, in order to check out some of the thoughtful commentary and dialogue that is ongoing there. Henry Farrell takes the time to consider and respond to the feedback.

But let me highlight a few of the passages in the beginning that outlines their perspective that resonated with me:

What are broad macro-institutions such as politics, markets and hierarchies good for? Different theorists have given very different answers to this question. The dominant tradition in political theory tends to evaluate them in terms of justice — whether institutions use procedures, or give results, that can be seen as just according to some reasonable normative criterion. Others, perhaps more cynically, have focused on their potential contribution to stability — whether they produce an acceptable level of social order, which minimizes violence and provides some modicum of predictability. In this essay, we analyze these institutions according to a different criterion. We start with a pragmatist question – whether these institutions are useful in helping us to solve difficult social problems

Some of the problems that we face in politics are simple ones (not in the sense that solutions are easy, but in the sense that they are simple to analyze). However, the most vexing problems are usually ones without any very obvious solutions. How do we change legal rules and social norms in order to mitigate the problems of global warming? How do we regulate financial markets so as to minimize the risk of new crises emerging, and limit the harm of those that happen? How do we best encourage the spread of human rights internationally? 

These problems are pressing — yet they are difficult to think about systematically, let alone solve. They all share two important features. First, they are all social problems. That is, they are problems which involve the interaction of large numbers of human beings, with different interests, desires, needs and perspectives. Second, as a result, they are complex problems, in the sense that scholars of complexity understand the term. To borrow Scott Page’s (2011, p. 25) definition, they involve “diverse entities that interact in a network or contact structure.” They are a result of behavior that is difficult to predict, so that consequences to changing behavior are extremely hard to map out in advance. Finding solutions is difficult, and even when we find one, it is hard to know whether it is good in comparison to other possible solutions, let alone the best. [Bold added]

Here are some direct parallels to the issues with schools that Will and I have been outlining on this blog. We have argued that schools, as Dewey stated, are primarily social institutions, and as such, they are highly complex, dynamic and dense with relationships and interdependencies.

We argue that macro-institutions will best be able to tackle these problems if they have two features. First, they should foster a high degree of direct communication between individuals with diverse viewpoints. This kind of intellectual diversity is crucial to identifying good solutions to complex problems. Second, we argue that they should provide relative equality among affected actors in decision-making processes, so as to prevent socially or politically powerful groups from blocking socially beneficial changes to the detriment of their own particular interests.

On the micro-level of school systems and schools themselves, these two features continue to bear saliency. Our schools and school systems are fundamentally hierarchies, linked up in a chain of command from the principal all the way up to the state governor and the federal government. Yet as many observers and advocates have been pressing on, these hierarchical structures often fail our students most in need, and the working conditions for teachers can be dehumanizing. The solution? For many new education reform advocates, markets seem to be the holy grail. But there are dangers in markets in solving complex social problems, as Henry and Cosma point out here:

On the one hand, free market participation provides individuals with some ability (presuming equal market access, etc.) to break away from abusive relationships. On the other, markets provide greater voice and choice to those with more money; if money talks in politics, it shouts across the agora. Nor are these effects limited to the marketplace. The market facilitates and fosters asymmetries of wealth which in turn may be directly or indirectly translated into asymmetries of political influence (Lindblom). Untrammeled markets are associated with gross income inequalities, which in turn infects politics with a variety of pathologies. This suggests that markets fail in the broader task of exposing individuals’ differing perspectives to each to each other. Furthermore, markets are at best indifferent levelers of unequal power relations.

Their advice?

This leads us to argue that democracy will be better able to solve complex problems than either markets or hierarchy, for two reasons. First, democracy embodies a commitment to political equality that the other two macro-institutions do not. Clearly, actual democracies achieve political equality more or less imperfectly. Yet if we are right, the better a democracy is at achieving political equality, the better it will be, ceteris paribus, at solving complex problems. Second, democratic argument, which people use either to ally with or to attack those with other points of view, is better suited to exposing different perspectives to each other, and hence capturing the benefits of diversity, than either markets or hierarchies. Notably, we do not make heroic claims about people’s ability to deliberate in some context that is free from faction and self-interest. Instead, even under realistic accounts of how people argue, democratic argument will have cognitive benefits, and indeed can transform private vices (confirmation bias) into public virtues (the preservation of cognitive diversity)6. Democratic structures – such as political parties – that are often deplored turn out to have important cognitive advantages.

In a sense, they are advancing an argument that the way much of public education currently stands in the media is in fact an advantage. We have a significant and growing amount of debate that is generated around education and education reform. As I’ve discussed before with Sara Mead, education reform should be messy. Democracy is a messy process. It would be so much cleaner, so much more efficient, if we had well-lubricated systems calibrated to deliver agreed upon knowledge and learning. But it’s important to keep the perspective on the bigger picture and acknowledge that we all benefit from deliberation and exposure to drastically different points of view.

However, anyone who works in a school or within the broader school system knows that at some point beyond deliberation, decisions must be made and acted upon, and that this process is far from democratic in application. A commentator, Jed Harris, points this out in a well-articulated comment on Crooked Timber:

Your argument doesn’t argue for democracy in any normal use of the word, but for social formations that don’t use power relations or exchange as primary constitutive elements.
Happily there are such formations and you give examples. Sometimes democracy (in the normal sense of the term) facilitates them, sometimes it is a captive of power relations and/or exchange relations and facilitates those much more.

Henry responds to Jed with this comment:

More generally – and this is a substantial weakness of our argument at the moment – we are all about democracy as a process of discovery, but have very little to say about it as a process of decision making. This needs to be beefed up, and will (probably not in this piece itself, but in its offspring).

So the key takeaway for me was that democracy is fundamental as a macro-institutional process of discovery and communication, but in terms of decision making and ground level implementation, other governance structures may be necessary. For example, in open source models of governance, benevolent dictatorships and meritocracies tend to be the typical form of ultimate decision making. One of the important facets of open source, however, is that if disagreements within the community are large enough, the original source code can be forked and developed in whichever way the new community members so choose.

What application do these insights hold for public education? I think this topic bears substantial relevance to the concept of unions, collective bargaining and the importance of bringing diverse perspectives to the table. I think it furthermore points to the utilization of technology to connect teachers directly to policymakers and include their voice and perspectives in the dialogue.

* [I’m the type who generally scavenges anything he can for free, but The Browser is one of the few online services I’m considering investing in a membership in — though I’m holding off until summer, as I already waste too much time trying to keep up with online media as it is.]

Let’s Get Tangled Up In Meaningful Reform

Sara Mead was a good sport and responded thoughtfully to my last blog post on how education reform should be messy.

Mark Anderson has some interesting thoughts here about the role of teacher voice in sustainable education reform. One of Mark’s comments is that people need to get more engaged in “messy issues” of education reform. And I don’t disagree with that; messy issues are messy in part because they have a lot of important stuff tangled up int hem. But you can’t engage a tangled, messy issue all at once, whether it’s at a policy level, at a school or classroom level, or something as personal and mundane as cleaning your house. You have to break it down into targets that you can change and leverage little victories on some of those targets into eventual bigger ones.
Mark’s also surely correct that ed reformers shouldn’t think we can “somehow sit from outside of schools and tweak external mechanisms and change the culture inside of schools.” But I’d challenge both the extent to which education reformers actually think that or some of the implications Mark draws from it. Policy types need to have real humility about what policy can and cannot do. To my mind, a big part of what policy reform needs to do is NOT try itself to change the culture inside of schools, but to eliminate obstacles and put in place conditions to enable effective teachers and leaders to do the real hard work Mark is talking about here (with accountability–and I mean that broadly, not in the specific terms it often takes in the current debate–as the natural and enabling counterpart to that). [Bold added]

I think there’s plenty I can agree with here (especially on the need for real humility–from all involved), but the devil is in the details. For example, when Mead states that you have to break up the messy issues into targets, the question that really matters here is “what targets?” The targets that policymakers have been mostly pursuing are not the targets that I believe will “put in place conditions to enable effective teachers and leaders” to do the real work of education reform. On the contrary, as Will and I have been saying, these targets have in fact traditionally been a hindrance to our work as educators. And a large part of the reason for that disconnect is that teachers have not been directly involved in the process of policymaking.

So there’s an obvious bridge that needs to be built here, and fortunately there seems to be a growing recognition that teacher voice needs to be meaningfully integrated into higher level decision-making processes. But progress along that front is slow, and meanwhile, educators and students are suffocating in the strictures of ill-considered, disassociated policies.

Will and I are forwarding the perspective of schools as ecosystems because we believe this model provides a focus on the targets that will have the greatest impact: the contexts and the content that schools create and deliver. But policymakers have been focusing on targets that will have little meaningful impact beyond political shell games, targets such as the accountability that Mead refers to above. Accountability is the natural offspring of professional environments and cultures, not the precursor. Instead of focusing on the hiring and firing of teachers, we should focus on curriculum. We should focus on scheduled and paid planning time. Focus on the pre-service and in-service training and support that principals and teachers receive. Focus on whether there is enough access to healthy spaces for students to play in and designated time for such. Focus on what students are being fed. Focus on the support and training parents have to raise their children. These are the things that will raise student achievement and enable equity.

Over the months that come, Will and I plan to demonstrate that our model is relevant and meaningful to both policymakers and educators. We hope that we can help policymakers and other stakeholders to see that it is the messy targets that are hardest to touch from a distance that are the most meaningful to pursue — and that the best way to pursue them is from the bottom up, not from the top down.

Education Reform Should Be Messy

Over on EdWeek, Sara Mead posted a thoughtful piece reflecting on a quote from Saul Alinsky’s Rules for Radicals and how it may apply to education reform. She writes that we need to focus on targets we know we can shift.

The United States has a shamefully high child poverty rate, and poverty and associated ills certainly impact children’s educational outcomes and make educators’ jobs harder. But once we start to focus on poverty it’s easy to get tangled up in a web of interconnected and deeply challenging issues that are very hard to address–including some we don’t fully understand or know how to address, and others where political and fiscal realities put bold action out of reach. So people who want to improve outcomes for low-income American kids today need to heed Alinsky’s advice and pick targets where we have the power to start.

Public schools, their policies, and effectiveness are a more promising target than the family, deep rooted neighborhood dysfunction, or “poverty” writ large. [Bold added]

From a tactical standpoint, Mead’s point makes sense. But I also think this perspective highlights a fundamental issue with education reform at the moment. It’s this idea that we can somehow sit from outside of schools and tweak external mechanisms and change the culture inside of schools.

We require a fundamental shift in our philosophy and approach to public education reform from the mentality of a “black box,” hands-off, top down approach that manipulates short-term external mechanisms to that of a problem-solving, sustainable perspective grounded in the grit of everyday classroom practice. Teacher voice must be directly involved in the process of policy-making, curriculum design, resource allocations, and other systemic facets that tie directly into the reality of everyday classroom experience.

What I do agree with is that schools possess the capability to provide equity for the disadvantaged, and that there is great potential to improve the quality of our public schools. So in light of that, I would advise all those who are passionate about public education to get your hands dirty in the soil that lies at the root of education. Learn from TFA’s NY Executive Jeff Li, who recently stepped down from his position to return to the classroom. Step foot into the classrooms, step foot into the schools, step foot into the communities that are struggling with poverty.

Instead of picking neat and easy targets, we should get tangled up in the “web of interconnected and deeply challenging issues that are very hard to address.”

Working inside schools and communities is the real work of education reform.