School Climate Matters

A classroom in Guipuscoa

Chalk up more research confirming what-we’ve-been-saying-all-along here at Schools & Ecosystems: a school’s learning environment impacts student learning.

In case you don’t know, NYC has been collecting what folks call “school climate” data via surveys administered to teachers, parents, and students since 2007. It’s important information to have about a school–arguably more important, to my mind, than test scores (I believe both should be considered).

Last July, I had quoted Match Education’s Mike Goldstein asking an important question about all this data:

Is anyone aware of scholars and reporters digging deep into this data set?  Is there any other data set in the USA just as good?

I think it’d be hugely productive to identify NYC schools which have made progress in “Total Climate” — and then study why.

Well, Mike, you’ve got your answer.

NYU’s Research Alliance for New York City Schools published a study using NYC’s school climate information that demonstrates that a school’s learning environment not only impacts student learning, but furthermore teacher retention. As Chalkbeat NY’s Alex Zimmerman reports:

Each measure, the report found, is independently linked to decreases in teacher turnover. And gains on two of those measures, high academic expectations and school safety, were directly connected to better scores on state math exams.

The study found that if a school improved from the 50th percentile across the study’s four measures of school climate (leadership, expectations, relationships, and safety) to the 84th percentile, teacher turnover would decline by 25 percent, or 3.8 percentage points.

A similar percentile increase in measures of school safety and high academic expectations alone boosted math scores enough to account for an extra month and a half of instruction. (Improvements in school climate also boosted language arts scores on state tests, but those gains weren’t statistically significant.)

It’s important to note that this study confined its focus to the following aspects of school climate:

  • safety and order
  • leadership and professional development
  • high academic expectations
  • teacher relationships and collaboration

Missing in such an examination (and mostly from these surveys themselves) is a focus on the physical environment of a school. There are questions pertaining to cleanliness and conditions of a school, but as we’ve also been arguing on this blog, the actual design, and the incorporation (or absence) of access to natural light and greenery, colors, furniture, etcetera (all largely subconscious factors), all have an impact on learning and relationships in a school.

If your school is interested in collecting school climate data, the US Department of Education is sharing free surveys and information for collection of data similar to NYC’s. Check it out and share.

Advertisements

The Power of Obliquity

M0015339 Indirect percussion method practised by Apache Indians
CC BY 4.0 (http://creativecommons.org/licenses/by/4.0), via Wikimedia Commons

We have a few more items to chalk up to the power of unconscious stimuli and bias on our feeble human minds:

  1. The basic shape of a logo is enough to bias your perception of that company’s product.

“Five experiments document that the mere circularity and angularity of a brand logo is powerful enough to affect perceptions of the attributes of a product or company,” the researchers write in the Journal of Consumer Research.

—”The Shape of a Logo Has a Powerful Impact On Consumers” on Association for Psychological Science

2. White college students associate black faces with violence and guns, even when those faces are as young as 5 years old.

“Our findings suggest that, although young children are typically viewed as harmless and innocent, seeing faces of five-year-old Black boys appears to trigger thoughts of guns and violence,” said lead study author Andrew Todd, an assistant professor of Psychological and Brain Sciences at the University of Iowa.

—”Faces of Black Children as Young as Five Evoke Negative Biases” on Association for Psychological Science

3. On a more positive front, conducting a value affirmation exercise can not only mitigate the “stereotype threat” for individual students who may be discriminated against, it furthermore seems to transfer that benefit to the entire class.

New research suggests that when students who are vulnerable to being stereotyped complete exercises that cause them to reflect on their own personal values, they perform better in class — and so do other students around them, even if those other students don’t complete the self-reflection tasks themselves.

—Jacqueline Howard, “Shielding Students From Stereotypes Helps Way More Than We Thought” on the Huffington Post

As the lead researcher in the latter study stated, “Who would have thought that the indirect benefits of an intervention could be almost as large than the direct benefits?”

Yet more evidence that speaks to the power of oblique methods of influence.

For more posts on this blog about obliquity, click here.

Friedrichs v CTA, and Thinking Probabilistically

By Matěj Baťha (Own work) [CC BY-SA 2.5 (http://creativecommons.org/licenses/by-sa/2.5)%5D, via Wikimedia Commons

Yeah, that headline was a mouthful.

But here’s the thing. You’re going to hear a lot of ed folks declaiming on the potential outcome of the Friedrichs v California Teachers Association SCOTUS case over the next few days. For good reason, as this is a case that may well prove to be more determinative of the future of public education in this country than ESSA.*

I’ve been reading Daniel Kahneman’s excellent Thinking, Fast and Slow lately**. Kahneman’s book is all about ideas we’ve touched on before here, such as cognitive bias and uncertainty. We’ve also looked at how “probabilistic thinking” could be used to overcome bias. So when I fortuitously came across this article on how “superforecasters” use probabilistic thinking, as well as a “base rate,” or “reference class” in order to make more accurate predictions, it jibed well with my understanding, and I think there’s useful lessons to heed as Friedrichs case is heard over the course of this week.

Rather than ideologically proclaiming sweeping predictions, as the experts are wont to do, “superforecasters” are less certain about their predictions, which ironically makes them better predictors. Professor Philip Tetlock delineates between “hedgehogs” and “foxes,” and notes that superforecasters are more akin to foxes:***

According to Tetlock, foxes are more pragmatic and open-minded, aggregating information from a wide variety of sources. They talk in terms of probability and possibility, rather than certainty, and they tend to use words like “however,” “but,” “although” and “on the other hand” when speaking. . . 

Unfortunately, most of the predictions you see in the media lack the specificity necessary to test them, like a specific time frame or probability, Tetlock says. . . 

Instead, Tetlock advocates for something he calls “adverserial collaboration” — getting people with opposing opinions in an argument to make very specific predictions about the future in a public setting, so onlookers can measure which side was more correct.

What does this have to do with Friedrichs? Well, I would suggest asking education “experts,” who will write about their ideas on the case, to assign a probability to their predicted outcome.

Based on my own, extremely limited understanding of the case, I think there’s a 65% chance that Friedrichs will win. I could well be completely wrong. But you’ve got my prediction here, in writing, with a timestamp on it, so you can hold me accountable to this.

I’ll write more on my thoughts on the case soon, but in the meantime, my thinking on Friedrichs v. CTA in a nutshell:

I think public sector unions need to change and adapt much more rapidly to a changing workforce and economy, but I believe strongly in the necessity for unions to present a necessary counterbalance to government and private financial interests. If Friedrichs wins, as I’m afraid she might, then we will witness a drastic further decline in the power of unions in our country. I believe this will be to the detriment of the long-term interests of our nation.

The only commentator I’ve seen thus far who’s beginning to think ahead to this outcome is Dan Weisberg of TNTP. He doesn’t assign a probability to the outcome, but implies it when he says the following:

Unfortunately for the unions, at least five Supreme Court justices appear to be more sympathetic to the teachers’ arguments than I am. The Court practically invited this challenge when it stopped just short of striking down agency fees in a similar case a few years ago.

I’m hoping our unions are already preparing for the worst, because no amount of impassioned op-eds can influence the outcome at this point.

*my apologies to all non-US residents for the US-specific jargon in this post.

**thanks to Deputy Chancellor Josh Wallack, who recently bestowed the book on educators at a dinner hosted by the NYC DOE Office of Leadership.

***We’ve looked at hedgehogs and foxes here before:

 

UPDATE 2/13/16:

Justice Scalia has just died, so that completely changes the odds. While I had first assigned a 65% probability to Fredrichs winning this case, my forecast has shifted closed to 40%. Read more on SCOTUSBlog: “The most immediate and important implications involve that union case.  A conservative ruling in that case is now unlikely to issue.”

 

In the US, the Experience of Poverty is Worse

“The researchers found that the relationship between genes, socioeconomic status, and intelligence depended on which country the participants were from.

“The hypothesis that the genetic influence on intelligence depends on socioeconomic status was not supported in studies outside of the US,” says Tucker-Drob. “In the Netherlands, there was even evidence suggestive of the opposite effect.”

The researchers suggest that the stark difference between the US and other countries might be explained by differences in how low socioeconomic status in experienced in the countries. That is, the relatively robust healthcare and social-welfare programs in Western Europe and Australia may buffer some of the negative environmental effects typically associated with poverty.”

—”Poverty Dampens Genetic Influence on IQ, in the US” on the Association for Psychological Science

This is fascinating. It corresponds with the idea that mindset and perception are deeply interrelated with poverty, which we also explored as a thesis of Scarcity. In the US, the experience of being poor is often equivalent to the experience of failure.

Yet another reason for a focus on social-psychological interventions.

Obliquity, Zen, and a Cultural Science of Schools

Target

Dominic Randolph, the head of Riverdale Country Schools, and Max Ventilla, the founder of AltSchool, have written a thought provoking series of letters on Medium, on the topic of “What’s the Point of School?

Randolph’s final letter, “Reimagining School,” contained two great concepts I wanted to especially highlight:

If schools took learning more seriously and aimed indirectly but intentionally, like a Zen archer, then I think they might be in quite different places.

This is an interesting way to phrase the principle of obliquity and pair it with the concept of social-psychological interventions! I agree strongly with Randolph on this. I tire of hearing leaders in education harp on shallow and direct sources of targets, such as test scores or standards or compliance rates, as if these are the ultimate drive or purpose in our work. Data and standards are only powerful when contextualized and analyzed from multiple perspectives, in consideration of the needs and interests of students and the curricular focus of the department and school. Compliance rates . . . well, that’s a baseline, not a target. Yet the messages I and other educators too often receive is that our focus should lie on such desultory targets.

How can we bring “applied anthropology” into schools so we can learn about how to more effectively construct the right culture of engaged and deep learning for our students, teachers, and parents?

People talk about the effect of a leader on a school environment or the way a particular class is difficult, but where is the science of the culture of schools?

Matt Candler’s ideas about “Tiny Schools,” in which small schools are begun as prototypical experiments, then brought to scale as they experience success, seems to be a great opportunity for such an expansion of “applied” science, most especially if the learnings from the failures of these experiments are shared transparently and openly.

I certainly hope there’s more thirst out there for such an education-specific science! I know I have this thirst myself. It’s hopeful that schools are pairing more often with academic researchers or design firms like IDEO, though I fear that such pairings end up focusing on schools and communities, like Randolph’s, that are already set up for success.

I’d perhaps be more interested in those pairings if they sought to expand the concepts of design thinking and character education to their broader community in the Bronx. How can school learning experiences build connections between students and schools of varying skills and backgrounds? And how can those connections be leveraged to actively overcome the physical and mental silos of our city and nation?

On Stealthy Interventions

By Sidharta-999 (done by a friend) [Public domain], via Wikimedia Commons
Last weekend I attended the Research ED conference, an intriguing new style of conference organized by Tom Bennett and other educators from across the pond. Getting the opportunity to listen and learn directly from educators I’ve only read online, such as Daisy Christodoulou, Carl Hendrick, and David Weston, was an enriching experience.

There’s many bits of learning I gleaned from that day, but there was one piece of research that really stood out to me in relation to the focus of this blog. Carl Hendrick, in his presentation with Harvard GSE’s Christina Hinton on a grassroots model of education research (view the presentation here), made an off-hand reference to the research of David Yeager on the concept of “stealthy interventions.” It piqued my interest, so when I got home, I dug up this review by Yeager and Walton, “Social-Psychological Interventions in Education: They’re Not Magic.”

As readers of this blog know, we’ve explored the concept of “obliquity” as critical to sustaining impact within a complex environment, especially in relation to goal-setting (see the following posts: 1, 2, 3, 4).  This is why the idea of a “stealthy intervention” caught my attention—that sure sounded like the principle of obliquity in action.

I’ve struggled with the application of the principle of obliquity within my own practice in the classroom—so I was excited to find that the “social-psychological interventions” outlined by Yeager and Walton are fairly concrete, once some resources and materials to apply them are developed. Some of them I’m already familiar with and have been applying—I’ve been teaching my students about a growth mindset since I first read Rita Smilkstein’s “We Were Born to Learn,” and I’ll use affirmation exercises with my students prior to state tests. But to wield such interventions more strategically and systematically, especially to assist students who are in crisis or struggling with significant academic hurdles, is something my special education department and I are seeking to build across our school. So I brought Yaeger and Walton’s review to my team, and we did a close reading of pages 274-275, which proffers insight such as the following:

“This analysis draws on a core tenet of social psychology, namely, that every attitude and behavior exists in a complex field of forces—a “tension system”—in which some forces promote a behavior whereas other forces restrain that behavior (Lewin, 1952; Ross & Nisbett, 1991; Wilson, 2006). One lesson from this analysis is that the structure of the system determines its potential for change—an intervention that increases students’ motivation to learn or that removes barriers to learning will improve academic outcomes only when learning opportunities exist in the educational environment.” [Bold added]

What I love about this exposition is how the authors explicitly acknowledge the complexity of the educational environment, then explain how oblique interventions operate within such an environment. The authors explain the success of a social-psychological intervention as attributable to how they fold into and reinforce the learning activities already present within the school (what they term “recursive processes”):

“. . . what can seem especially mysterious is how a time-limited or one-shot social-psychological intervention can generate effects that persist far ahead in time. For instance, people may assume that an intervention has to remain in mind to continue to be effective. But like any experience, a psychological intervention will become less focal as it recedes in time. As we suggest below, a key to understanding the long-lasting effects of social-psychological interventions is to understand how they interact with recursive processes already present in schools, such as the quality of students’ developing relationships with peers and teachers, their beliefs about their ability, and their acquisition of academic knowledge. It is by affecting self-reinforcing recursive processes that psychological interventions can cause lasting improvements in motivation and achievement even when the original treatment message has faded in salience (e.g., Walton & Cohen, 2011).” [Bold added]

In other words, social-psychological interventions leverage relationships and the experience and knowledge of students and school staff. That’s what I call viewing a school as an ecosystem.

The most powerful takeaway I got from this research, aside from concrete suggestions for activities and interventions based on their review, was the heuristic of designing an intervention or goal-setting process that accounts for the student’s subjective experience:

“. . . social-psychological interventions can be brief yet impactful because they target students’ subjective experiences in school and because they rely on a rich tradition of research on persuasion and attitude change to powerfully convey psychological ideas.” [Bold added]

As my special education department seeks to include our students in the process of setting their IEP goals, this is a good rule of thumb to bear in mind.

Cuomo: Build Relationships, Not One-Measure-to-Rule-Them-All

By Codking (Own work) [Public domain], via Wikimedia Commons
In my last post, I explained why NY Governor Cuomo’s push for greater weight on testing was bad strategy. In a study in the American Journal of Public Health on schools with Zero Tolerance policies, here’s some further confirmation that linear thinking backfires in complex environments:

The authors found that “students attending schools with suspension policies for illicit drug use were 1.6 times more likely than their peers at schools without such policies to use marijuana in the next year.” That result held for the student body as a whole — not just for kids who were suspended.

There was another interesting result discovered in this study: relationships are better medicine.

But they did find one factor that actually did seem to decrease the likelihood of drug use: student-teacher interactions. At some schools, students caught with drugs were sent to their teachers to discuss the consequences of drug use. This was associated with a 50 percent decrease in the odds of later marijuana use.

What’s the lesson here for state policy on education?

Don’t seek to impose brute force policies to cover up poor politics and inadequate funding and the most segregated schools in the country. What will really make a difference in education? How about building meaningful relationships with practitioners and the folks on the ground who work in communities and schools facing the greatest of challenges? How about providing them with real funding that demonstrates that we value them?

In the face of complexity, policies that seek to foster empathy and community will be the ones most likely to sustain change.

Source for quotes in this post:

Christopher Ingraham, “Zero-tolerance school drug policies only make drug use worse, a study finds” on The Washington Post, via link on Twitter from RealClearEducation.

Cuomo Doubles Down on Linear Thinking

“The dunes here are linear, thought to be due to shifting wind directions.”

New York’s Governor, Andrew Cuomo, has decided to double down on a misguided effort to increase the weight of state assessments in a teacher’s evaluation from 20% to 50%.

I’m going to spare you the “corporate reform” and “hedge fund buddies” angle on this; instead, I contend that this single-minded focus on test scores is simply bad strategy.

The focus of this blog has been on teasing out the metaphor of a school as an ecosystem, and elaborating on the theme that managing complex systems requires moving beyond linear thinking.

Here’s a relevant quote from Steve Denning (which we’ve examined before) on the principle of obliquity:

Efforts to impose linear thinking on complex situations have often led have the opposite of what was intended. As a result, the principle of obliquity becomes relevant. Where explicit articulation of a goal will result in the complex environment pushing back in the opposite direction, oblique goals will often be more effective, e.g. the goal of delighting customers may make more money than an explicit goal of making money.

What would be the opposite of what was intended in this situation (if Cuomo gets his way)? Well, if every teacher in the state is conscious that their evaluation is heavily determined by their student’s performance on that state test — then the problems of focusing mostly on ELA and math and shallow skills-driven test-prep will most likely be exacerbated. And kids that most need access to rich literature and knowledge across the domains of history, music, arts, science, and technology will instead continue to be given drivel. And teachers and schools may be more likely to engage in cheating.

I generally assume best intent when assessing the decisions of others. So to be fair to Cuomo, he is pushing for a simplified accountability system because the current system of 20% state, 20% local, and 60% principal observation may lead to the problems of over-testing and inflated scores.

But moving to increase the weight of the state test scores as a leverage over teachers is not the right move to resolve these issues. Instead, this maneuver is much more likely to compound deeper issues, rather than achieve the goal of increasing student and teacher performance.

It would be great if our elected representatives could move beyond linear models when making critical decisions on how to improve our system of education.

Obliquity Strikes Again: Oblique Messages Better Meet Their Targets


“The results were clear: While higher depression scores were generally linked with lower intentions to seek help from a romantic partner, a close friend, or family members among participants who received the direct message, there was no such relationship for the participants who received a mistargeted message.

According to the researchers, these findings indicate that the mistargeted message was more effective at promoting help-seeking than the direct message was. And a second online study confirmed this pattern of results.” [bold added]

–“Mistargeted Messages Could Spur Help-Seeking for Depression” via Association for Psychological Science’s blog, Observations

Click here for more on the principle of obliquity

The Luxury of Efficiency

By Crops for the Future (Own work) [CC-BY-SA-3.0 (http://creativecommons.org/licenses/by-sa/3.0)%5D, via Wikimedia Commons
I’m currently reading Jared Diamond’s The World Until Yesterday, in which he examines traditional societies and compares them to modern state societies, and I came across a passage which lends an interesting perspective to a word Will brought up recently: efficiency.

Diamond describes a traditional form of planting that has greatly perplexed modern minds: field scattering. Why are these ignorant peasants and tribal peoples wasting their energy in planting and tilling and traveling between many small plots, rather than consolidating their yields? “To modern economic historians, that was “obviously” a bad idea.”

It turns out that it has a lot to do with managing complexity in the face of immediate needs.

“In any given year there are big differences between yields of different fields, but a peasant can’t predict which particular field is going to produce well in any particular year,” Diamond states of the Cuyo Cuyo farmer.

So in the face of the unknown, the peasant scatters his potential yield. This decreases his overall yield, and ensures he will rarely have a great abundance (which he wouldn’t be able to store for long anyway), but also ensures that he will rarely starve.

“If your time-averaged yield is marvelously high as a result of the combination of nine great years and one year of crop failure, you will still starve to death in that one year of crop failure before you can look back to congratulate yourself on your great time-averaged yield.”

Thus, “through long experience, and without using statistics or mathematical analyses, Goland’s Andean peasants had figured out how to scatter their land just enough to buffer them against the risk of starvation from unpredictable local variation in food yields.”

I wonder what lessons there may be in this for schools.

In the face of stark accountability (“starvation”) and potential closure, a school may strategically “scatter” its efforts to meet its immediate needs, rather than “efficiently” investing in more coherent and systematic measures that will, over time, accrue in more lasting impacts and yields.

After all, when you are facing starvation, your primary concern is not to starve. But when you already have a buffer of wealth, you can take greater risks.

We often talk about how much money America spends on education in comparison to other countries, with little to show for it. Yet school funding is often on a perpetual cycle where schools are encouraged to scatter their money willy-nilly on immediate needs, rather than take bigger risks and strategically invest in longer-term investments.

I’ll stop there, as I’m probably pushing the analogy too far. But it’s interesting to consider how the concept of “efficiency” can be considered as a product of luxury.