Charter vs. District Systems

By NASA’s Aqua/MODIS satellite (http://visibleearth.nasa.gov/view_rec.php?id=6204) [Public domain], via Wikimedia Commons
Neerav Kingsland looks at the recent findings on professional development via the TNTP Mirage report and the Rand Corporation study, and comes to the conclusion that “Professional development only seems to lead to student achievement increases in charter schools!”

I noted in a recent post that in the TNTP study, teacher effectiveness and growth was notably more observable in a CMO, and I hypothesized that this could be attributable to some charter networks having more tightly managed systems of assessment, curriculum, teacher practice, and observation.

But to suggest that this is an innate quality of charter schools is questionable. There is absolutely no reason for a district school not to be in possession of such qualities, and indeed, many do.

Kingsland argues for NOLA-style systems, in which the government merely regulates, rather than operates, schools, with the idea being that the private sector can conduct operations more efficiently and effectively. But there’s a potential, and possibly critical, issue with such a system: a lack of coherency.

Within a well-managed district, on the other hand, there is potential for greater coherency. A state or central office can provide specific direction on operational shifts via policy that all district schools would be expected to adhere to.

Kingsland asks, “is it more likely that we can achieve major gains in districts or scale highly effective charters?,” I think he’s created a false dichotomy. I think the more interesting question is, “How can we achieve major gains by leveraging federal, state, and district policy to implement effective and coherent systems, content, and practices across all schools?”

A NOLA-style system might be able to make swift initial gains, due to well-managed networks putting into place strong systems of assessment, feedback, and practice. But it’s certainly feasible that a well-managed district system can make even bigger gains over the longer haul.

I disagree, therefore, with Kingsland’s position that charter schools are inherently superior in enhancing teacher effectiveness and promoting student achievement. In fact, I charge that a NOLA-style system may ultimately run up against its innate incoherency, at which point, large-scale gains would stagnate.

I could be totally wrong on this, of course, and admit that this is conjecture and based on my own values. It may be that a NOLA-style system may end up leading to greater coherency in operations due to competition, and thus, best practices evolve through demonstrated gains in one organization and subsequent adoption by those who are attempting to compete.

I may also be overstating the ability of district schools to establish coherency, given constraints in operating within often volatile political contexts.

The problem is, of course, that while NOLA has demonstrated significant academic gains on tests since moving into a private sector operated system, it’s still purely conjecture as to whether the same benefit would transfer to any other district simply due to a  structural change. It’s also still conjecture that those gains can be solely attributed to a structural shift to private sector operation, rather than the simple mechanism of distributing students across geographical boundaries.

But let’s assume for the moment that Kingsland is correct that a private sector operated school system is the optimal system. I would still argue, even in such a case, that this doesn’t mean that such a system will necessarily scale effectively into different social and political contexts.

In the face of great complexity and uncertainty, we can hedge our bets by planning for robustness, rather than optimality.

The question therefore becomes: what is the most robust? A school system operated by the public, or a school system operated by the private sector?

Perhaps the answer lies somewhere in between.

Advertisements

The Mirage of a Mirage: On the New TNTP Report on PD

IMG_20150718_135114

Today I went down to DC* for TNTP’s release of their Mirage report on the (non)impact of professional development on teacher effectiveness. TNTP presented an overview of the findings, then journalist Amanda Ripley facilitated an interesting panel with USED’s John King, teacher Jennifer Corroy Parras, DC’s Kaya Henderson, and AIR researcher Dr. Michael Garet.

The big story is that teacher PD is 1) enormously expensive, yet 2) there’s no silver bullet. In other words, teaching is complex (glad ed reform world is beginning to acknowledge this), so we still are very far from knowing what improves teacher practice at scale, yet we’re investing a lot of time, energy, and money in it willy nilly.

There are a lot of aspects to explore, and the panel touched on a number of them. Ripley provided some nice framing of the systemic issues, such as that most PD (and US system of ed at large) is disjointed and incoherent, and that there is a general lack of clarity around what effective teaching looks like, accompanied by high expectations. She noted that this lack of clarity and high expectations for teachers parallels student experiences in our educational system, with both teachers and students leveling off in performance after an initial 5 years.

Both John King and Kaya Henderson pointed out that an oft missing link in PD is a focus on curriculum, a point on which I strongly concur. Curriculum provides the frame, the backbone, the structure that daily guides student and teacher experiences in the classroom. Without that structure and purpose, we are left with the incoherent gloop that Ripley first referred to.

TNTP’s report is incredibly insightful and useful–if enervating–material. I feel like it’s also important to define the kind of PD which the report mainly seems to abide by. I’ll admit that I haven’t yet dived too deep into the report, but from an initial scan of the survey questions and findings, I obtained the distinct impression that they mostly mean “PD” as in a specific dosage or frequency of a prescribed intervention that results in a change in an individual’s teacher practice within the classroom (and which should then ideally translate into improved student outcomes).

When I think of PD, I think of teachers sitting together around a table and collaborating to design common assessments, curriculum, resources, and examining student work and data. This was a component of TNTP’s teacher survey (noted as informal & formal collaboration, and peer time), but the report notes that an equal amount of such time was dedicated to this in both “improvers” and “non-improvers.” But I think there’s one key aspect of such collaborative time that we need to delineate, and which I believe could distinguish between those two populations: how teacher collaboration and planning time connects to a school-wide system.

Let me give some specific examples of what I mean to make this tangible:

  • A bi-weekly 8th grade-level team meeting working to horizontally (within the grade) align curriculum and assessments and strategically build  and reinforce content knowledge and skills
  • A weekly ELA department team meeting working to vertically align curriculum and assessments (and strategically build  and reinforce content knowledge and skills)
  • A PBIS system of agreed upon (developed both school-wide and via grade-level teams) infractions, consequences, and rewards
  • An established vision for utilizing advisory time to provide social-psychological interventions, accompanied by planning time to design these interventions, and time to analyze and assess their impact

You could go into both an effective and ineffective school and witness such use of collaborative time–but the key is how that teacher collaboration ties into a coherent, school-wide system.**

Admittedly, though, this all goes back to the great complexity not only of teaching, but of working in a school. There are so many variables behind what goes into anything remotely approximating either “PD” or “teacher practice” that it is difficult to define what we’re looking for, and even more difficult to do well consistently and to replicate and scale across different contexts.

I came back to NYC with three takeaways that best summarize my thoughts from the TNTP report . We would do well to consider the following in education reform:

  1. Coherency (in curriculum & assessment, first and foremost)
  2. Clarity (in expectations, feedback, and roles for teachers and students)
  3. Hard work (and more hard work)

*Thanks to Educators 4 Excellence for this opportunity

**I think this is what may have contributed to the differences between the charter and district schools in this study. The charter network was more effective in supporting their teachers effectiveness and growth over time. Charter schools often have much tighter and coherent school-wide structures and systems tied to assessment, curriculum, teacher practice, and observation.

Why the new ESEA won’t change US education

12de7-statescolor

I haven’t been following the ESEA revisions closely as they’ve meandered their way through the Senate and the House, but my general impression is that this new bill will not dramatically transform public education in the United States (surprise!).

My understanding is that revisions as they currently stand are looking to:

The unfortunate fact is that perennial political debates between federal and state rights assumes precedence over the practical exigencies of public education. Transforming an entire nation’s education system requires strong federal guidance and authority.  The most current iterations of ESEA suggests that federal authority will be yet further curtailed.

This isn’t to say that NCLB was perfect. But the federal government clearly established higher standards and accountability to those standards for all students. This was critical for students that had been historically denied access to a quality education, such as students with disabilities. Yet NCLB was admittedly inflexible and created the unintended consequence of a single-minded focus on superficial ELA and math tests that were largely disconnected from any viable curriculum.

As problematic as the testing regime has been, however, tests could serve a more proactive instructional purpose if they were better designed to acknowledge the importance of content knowledge in literacy, if they were better able to adapt to student ability, and if they incorporated community feedback in the selection of the texts they assessed*. Then tests could actually provide direction on curriculum and instructional decisions. I’ve made this argument before, and so has David Steiner, former education commissioner of NY State, just so you know I’m not crazy.

I appreciate that senators are pushing for the creation of “an evidence-based innovation fund,” but I fear that such funds most likely won’t be put towards building infrastructure, despite a growing body of research demonstrating the impact of environment on behavior and learning.

I also greatly doubt that ESEA will be looking to fund or create accountability for any efforts at integration of schools or communities based on socio-economic status, despite evidence demonstrating its impact on reducing the opportunity gap.

I applaud our public representatives for working together across the aisle to negotiate these revisions of NCLB. But I am afraid that their efforts will do little to better outcomes for the majority of our students. Our nation needs to get real about increasing federal authority over curriculum, funding, and regulating school choice.

Let me be clear that I don’t mean that I’m advocating for one-size-fits-all. I believe states and districts require greater flexibility and choice in adapting funding and policies to their students’ needs. But without strong guidance and accountability from the federal government, states will be highly unlikely to do the heavy moral and financial lifting of pushing for more integrated schools and neighborhoods, designing or adopting tests that are adaptive yet more rigorous and content-rich, creating new school buildings that increase access to natural light and greenery, or pushing for increased access to a quality school and program and teacher for traditionally ignored populations.

*Update: I should add that tests should further be tied to rigorous, common standards.

Cuomo Doubles Down on Linear Thinking

“The dunes here are linear, thought to be due to shifting wind directions.”

New York’s Governor, Andrew Cuomo, has decided to double down on a misguided effort to increase the weight of state assessments in a teacher’s evaluation from 20% to 50%.

I’m going to spare you the “corporate reform” and “hedge fund buddies” angle on this; instead, I contend that this single-minded focus on test scores is simply bad strategy.

The focus of this blog has been on teasing out the metaphor of a school as an ecosystem, and elaborating on the theme that managing complex systems requires moving beyond linear thinking.

Here’s a relevant quote from Steve Denning (which we’ve examined before) on the principle of obliquity:

Efforts to impose linear thinking on complex situations have often led have the opposite of what was intended. As a result, the principle of obliquity becomes relevant. Where explicit articulation of a goal will result in the complex environment pushing back in the opposite direction, oblique goals will often be more effective, e.g. the goal of delighting customers may make more money than an explicit goal of making money.

What would be the opposite of what was intended in this situation (if Cuomo gets his way)? Well, if every teacher in the state is conscious that their evaluation is heavily determined by their student’s performance on that state test — then the problems of focusing mostly on ELA and math and shallow skills-driven test-prep will most likely be exacerbated. And kids that most need access to rich literature and knowledge across the domains of history, music, arts, science, and technology will instead continue to be given drivel. And teachers and schools may be more likely to engage in cheating.

I generally assume best intent when assessing the decisions of others. So to be fair to Cuomo, he is pushing for a simplified accountability system because the current system of 20% state, 20% local, and 60% principal observation may lead to the problems of over-testing and inflated scores.

But moving to increase the weight of the state test scores as a leverage over teachers is not the right move to resolve these issues. Instead, this maneuver is much more likely to compound deeper issues, rather than achieve the goal of increasing student and teacher performance.

It would be great if our elected representatives could move beyond linear models when making critical decisions on how to improve our system of education.

Hi-Lab: A Test of Learning Aptitude

Might we one day take a test that determines whether we require our teaching to be more or less explicit in nature?

“In one study published in Psychological Science in 2013, people’s performance on an implicit learning task correlated with an ability to learn Hebrew words, and in another study published in Language Learning, people who did well on sequence learning tasks (such as those used in the Hi-LAB study) also did well understanding grammar in a second language. It all points in one direction: “that implicit learning may be a large part of how second languages are acquired,” said Bozena Pajak, a postdoctoral researcher at the Center for Language Sciences at the University of Rochester. . . .

The Hi-LAB can also become a powerful tool in the educational sphere. Because it identifies a person’s learning strengths and weaknesses, the test results could grease a student’s instructional path by showing how they learn best, whether it’s through reading or talking to people, through explicit grammar lessons or watching movies. CASL executive director Amy Weinberg said that a partnership is in the works with IBM to develop a learner management system that would deliver the appropriate language learning material to people based on their test results and other factors.”

–Michael Erard, “Secret Military Test, Coming Soon to Your Spanish Class” on Nautil.us

How is Carmen Fariña doing from an “Ecosystems” perspective?

Curved arrow

As Carmen Fariña begins her first full school year as NYC chancellor, now is a good time for us to step back and reflect on how Ms. Fariña is doing from a socio-ecological perspective of leadership.

What might such a perspective of leadership entail? Glad you asked! Looking back through common themes we’ve explored on this blog, some relevant criteria that emerge could be as follows:

A leader who recognizes schools as ecosystems . . . 

  • Values inclusion and diversity (1, 2, 3, 4, 5, 6, 7)
  • Consistently observes local conditions (1, 2, 3, 4, 5)
  • Plays the long game  (1, 2, 3, 4, 5)
  • Models active listening (1, 2, 3, 4)
  • Applies intensive management (1, 2, 3, 4)
  • Displays a willingness to try different things (1, 2, 3, 4)
  • Utilizes the principle of obliquity (1, 2, 3)
  • Sweats the small stuff (1, 2, 3)
  • Demonstrates humility (1, 2)
  • Facilitates the confrontation of the brutal facts (1, 2)

We could keep going deeper into the sort of systems and investments such a leader might make, such as a focus on developing collaborative relationships, building in redundancy and robustness, creating greater optionality, investing in initial conditions, and investing in infrastructure, but just in terms of leadership, I think this provides us with a good start.

So by the aforementioned criteria, how is Carmen Fariña doing as a leader of NYC’s hugely complex school system?

Here’s what Ms. Fariña has done thus far in her tenure as chancellor:

  • Made parental engagement one of her top priorities.
  • Focused on elevating the role of the arts and extracurricular activities in schools.
  • Constantly stepped foot into a variety of schools, focusing on concrete feedback to school leaders, rather than needless politicizing.
  • Removed letter grades from school progress reports, making progress reports based primarily upon quality reviews from actual observation and contextual knowledge, rather than decontextualized data points.
  • Implemented a series of pilots throughout the city to test out new initiatives.
  • Demonstrated a vision for the sustainability of the profession by requiring longevity and experience for leadership roles in schools and districts.
  • 2014-15’s Citywide Instructional Expectations establish a continuum from prior CIE’s, rather than throwing out the baby with the bathwater. Continues to push for the positive intent and implementation of the Common Core.
  • Collaborates deliberately and strategically with the teacher’s union.

In sum, Ms. Fariña is shaping up to demonstrate the qualities of a leader who recognizes schools as complex systems and is able and willing to both intensively manage, while simultaneously maintain flexibility and empathy. Her actions and words thus far align well with the criteria of a leader with a socio-ecological mindset.

There may things going on politically behind the scenes at Tweed I don’t know about, and I may not agree with all of her positions, most especially her obvious coziness with Teacher’s College Reading and Writing Project and stress on independent reading. But Fariña’s leadership has palpably shifted the tone in NYC, and I’m excited to see how her initiatives will continue to develop and play out, and hopefully she will continue to provide a positive model to other leaders across the country.

NY State Commissioner John King on Poverty and Test Results

“As you can see, although there is a relationship between poverty and performance, there are exceptions at all levels of wealth. Just as there are schools that perform above and below the statewide proficiency level at lower levels of wealth, there are schools that perform above and below the statewide proficiency level at higher levels of wealth. . . .

“These results make clear that those who claim that demography is destiny and that we cannot improve teaching and learning until we have first fixed poverty are simply mistaken. In New York, there are many examples of higher poverty / higher performance schools . . . 

“This is not to imply that poverty is an unimportant factor – it is extremely important, for all of us. But the idea that poverty or family circumstances outside of school are insurmountable obstacles for teaching and learning is a fallacy. As educators, we should all be active in the national discourse on issues of inequality and how best to expand opportunity for all. However, we must commit ourselves to use the time we have with our students in school as effectively as possible and to do all we can to ensure that education helps to shape a path out of poverty. Our colleagues have done it. Our students have done it. We can do it.

“We need to understand the factors that help a school achieve better learning outcomes for high needs students (higher poverty / higher performance schools). Conversely, we also need to understand the reasons why other schools do not perform as well as their demographic peers, despite having an abundance of resources and wealth (lower poverty / lower performance schools). What are the policy, leadership, and instructional practices that produce great results for our kids that can be echoed and expanded across the state? What are the educational investments – from high-quality pre-K to expanded learning time to community schools partnerships providing wrap-around services to socioeconomically integrated magnet schools – we need to make as a state in order to accelerate improvement?

“That is our challenge – to understand how to keep getting better.”

–NY State Commissioner John King, in reference to NY test results in a message in an EngageNY newsletter

 

What Is Success?

Ecoschool

In an interesting coincidence, Will and I have been writing about Eva Moskowitz and poverty, and here the NY state test results come along. Success Academy made up 7 out 15 of the top scorers in NY state, according to the NY Post. And while of course poverty level correlated with reading performance, there were the outliers of high poverty schools that outperformed state averages.

But Success Academy was more than just a freak outlier—it blew other schools out of the water, and it did so consistently across multiple schools.

In the NY Daily News, Robert Pondiscio writes in “How Does She Do It?“:

What is imperative now is for serious, unbiased experts and observers to descend on Harlem and figure out how these extraordinary results are being achieved and, if all that glitters is gold, how to replicate them.

It’s worth repeating a challenge I threw out earlier to Eva Moskowitz in a recent post: share the practices, content, and protocols your schools are using so others can benefit. That’s what being a “public” school is all about, right? Collaboration, sharing, learning. That’s what our public system of education should be doing if we’re truly dedicated to improving outcomes for all of our children, and not just some.

As John King also said: “The question becomes, what’s happening in these schools that’s leading to those better outcomes?”

I agree with Pondiscio and King. I genuinely want to know what’s happening at Success Academy. And I want to know from the perspective of a school as an ecosystem. From this perspective, some questions I would ask would be oriented around the main pillars of a strong school community: 1) leadership, 2) content, and 3) environment.

Leadership

  • What do the formal and informal leaders say and do? How and what do they communicate consistently? (This includes student leaders).
  • Is the leadership distributed?
  • What mechanisms are in place for students, parents, teachers, and leaders to collaborate and receive continuous feedback? How do leaders respond to feedback?
  • How is diversity in student ability, knowledge, and skills strategically recognized and cultivated?

Content

  • What are the values and vision behind assessment and unit design?
  • What texts are taught in ELA? Why?
  • How well do topics and themes build knowledge and understanding of academic domains and the world sequentially across classrooms and grades?
  • How are students engaged in their community through units?
  • What scaffolds and interventions for students who are struggling are applied consistently both in and out of classrooms?
  • What opportunities beyond academics are provided for all students?

Environment

  • What does it feel like when you walk into a Success Academy school? What does it sound like? What does it look like?
  • How relevant is posted work and displays to students and their community?
  • What is the ratio of positive to negative language used by students and staff in the building?
  • How (psychologically) safe do students with special needs feel in the hallways, lunch rooms, and classrooms?
  • How are supportive social relationships and networks developed and sustained by the school?

These are just some of the questions I would start with.

State test results are just one indicator. Remember, we’re playing the long game here. And there is evidence to suggest that test results are a shallow indicator.

I don’t say this to put down what Success Academy has accomplished. Their students deserve our recognition. They’ve worked hard for this.

But what I want to know is whether what Success Academy is doing is truly preparing students for the future, for the long-term.

What is success? And what does it look like? And is what Success Academy doing truly an exemplar?

And more importantly—what does success look like over the long haul?

Grading without Grading

report card

A few months into my first year at an Upper Manhattan private school, I have finished my first round of report cards. Actually, at my new school we don’t call them report cards; we just call them “comments.” Why? Well, because that’s what they are: comments. Words. Not a single number on the page.

In other words, my students do not receive grades—or at least grades as they are commonly understood in the world of education. There is no number or letter designated to represent their academic performance. There is no top or bottom 10 percent. At a school where New York’s elite and powerful fight to get their children admitted, student progress is not measured quantitatively or competitively.*

To be honest, I’m still wrapping my head around this. After teaching in the public schools, where quantitative assessments determine student success and school funding, I’m still trying to figure out how the lack of conventional grades alters a school environment. For now, I simply think it’s worth noting that one of the most selective, respected schools in New York City relies entirely on qualitative assessments until its students reach their teens. As the Department of Education pushes scantron tests on students too young to hold their pencils, this seems, at the very least, worth our consideration.

*This is true through elementary and into middle school. From late middle school into high school, the students are graded. I teach sixth grade, so I only write comments.

Student Centered Data: Part IV

RRZEI Icon Set


In my last post, I proposed that data collection and reporting should be as seamlessly integrated into a teacher’s workload as possible. I also suggested that little time should be spent examining multiple choice results, and that instead, teachers should spend more time performing deep group analysis on student work across content areas through structured professional dialogue.

For my final post on this admittedly mundane but necessary topic (given its prominence), I will examine the following principle:

  • Data reports must be easily shared
Given that the world has recently discovered that the NSA is collecting a pool of metadata on our every online transaction, it’s all the more charming just how incredibly difficult it can be to extricate student information from “official,” proprietary systems of data collection. For example, here in NYC, we use  a well intentioned but overwrought online system called SESIS to create and house Individualized Education Programs (IEPs) for students with disabilities. Understandably, as these are legal documents, student privacy is of utmost concern and compliance with federal and state regulations is strictly adhered to.
However, there is a point where an IEP must not simply be written, but must be implemented. This is where it becomes incredibly difficult for a teacher to easily assess and track student progress towards IEP goals. Why? Because having an IEP on paper (which must be kept under lock and key) doesn’t facilitate easy access to student goals, and neither does having the IEP locked up in the online system, requiring time consuming navigation of convoluted pull down menus and screens to arrive at each specific goal.
There are well-meaning folks out there, such as the ones at Goalbook, which aspire to help educators better implement IEPs. But unfortunately, the process of transferring a student’s IEP goals from SESIS to a system like Goalbook (or a Google spreadsheet or whatever) is extremely time consuming. I know, because I did it. It took me well over a month of my free time (yes, I spent some of my winter break doing this) to go into each individual student’s individual goals and copy and paste (then edit to correct for altered formatting) into the other system. At the end of the day, I had all the student’s IEP goals from my school in one online tracking system! Great!
Problem was, I couldn’t force my other special education teachers working with me to use the system, nor do I blame them for not having the time nor inclination to jump into yet another online portal. So my exercise was largely pointless, other than as a lesson to me which I am now sharing with you.
While a student’s information should always be strictly confidential, keeping their data locked up in proprietary systems does little to aid either the student nor the teacher struggling to help them.
At some point, that information must be exportable at the discretion of the teacher, and it must be exportable in multiple formats that will allow for operability with a variety of other systems.

And of course, students and parents should have open access to their data at all times.

Seems like common sense, but unfortunately, as I mentioned in my last post, many data systems used in schools are not designed with end users in mind.

That’s a Wrap
To review the points which I’ve made regarding data in public education, let me review the three principles we’ve explored and see what key takeaways I can leave you with.

Just to clarify, in case the title of these posts hasn’t tipped you off, the point of all these recommendations is to frame the use of data in schools as to how it can most benefit students, as opposed to please adults who love colorful data reports and torturing each other by talking about data at meetings.

Key takeaways, with helpful highlights in bold and cute slogans on top:

Pay attention to the students in front of you
Any technology that is used for the purpose of data collection must be as hands-off and eyes-on as possible.

Current technologies which are mostly hands-off and eyes-on:

  • Paper checklists 
  • Video cameras 
  • Audio recordings 

Authentic, meaningful, and relevant data
The best data to examine is real student work from classroom assignments, not multiple choice test data.

Acknowledge superficiality

If we must examine multiple choice test data, automate the data collection and reporting process as much as possible so that analysis can be swift.

Real conversations about real students
The best method to exam student data is in the context of professional dialogue between teachers of different content areas.

Free range data
Allow data to be freely exportable, within reasonable limits with respect to privacy, of course.

Just a reminder
Technology is not a panacea. Especially in the realm of education. Rinse and repeat.

Was this boring series of posts on data useful at all to you? Please let me know if so, it would cheer me up greatly and help me feel that this wasn’t a pointless exercise in bloviation. Back to our regular program of Schools as Ecosystems, and piiiiiigs iiiiiiiin spaaaaaaace. . .