Conceptual vs. Procedural Math at Mastery charters

13946605527_82e5a3a848_z

“Maths” by Chris de Kok is licensed under CC BY 2.0

There was an interesting recent the74 piece on a Philadelphia charter organization, Mastery, which takes low performing schools and works to “turn them around.”

 

Embedded within this article is the implication that a shift to a focus on the teaching of conceptual math, rather than “rote” procedural teaching, led to a swift downturn in math scores.

“So this year, the network began reintroducing teaching techniques that had been a staple at Mastery schools for years, while seeking a middle ground between no excuses and restorative practices. It’s a ‘journey of trying to find out what’s the right mix,’ Gordon said.

Specifically, the network is reintroducing procedural math instruction, which focuses on rote instruction like memorization and repetition.”

It seems worth digging into this supposition a bit more.

Is Mastery’s downturn in math scores due to the failure of conceptual math in general as a pedagogical approach? Or is it a failure of the network to attract and train teachers who can teach this type of math more effectively?

Or is it a failure in the assessments that were used as a reference? Or was it that conceptual math takes longer to “stick” and pay dividends? Or was it a failure of the curriculum they used to move in a more conceptual direction? . . .

http://the74million.org/article/at-philadelphias-mastery-charter-network-culture-is-key-to-turning-around-failing-schools

Advertisements

Success can’t be measured by one or two numbers

“Whenever you make huge decisions about complex situations based on one or two numbers, you’re headed for disaster — especially when those numbers can be gamed.”

—Mark Palko and Andrew Gelman, “How schools that obsess about standardized tests ruin them as measures of success” on Vox

We’ve questioned Success Academy’s “success” on this blog before. These statisticians bring a new lens to that question.

I don’t want to denigrate the good work that Success Academy teachers and students are doing. There are practices and systems well worth replicating and investigating in these schools. But Eva Moskowitz’s political framing and marketing of her schools as the solution to poverty is problematic.

My current views on testing, in answer to my past views on testing

While up in Albany a few weeks ago, I was interviewed by someone from NYSED about what I might say to parents who are considering “opting out” their child from state testing. You can view the video here*.

Someone on Twitter, “WiffleCardenal,” voiced a critique to me regarding the video, in contrast to things I’ve said in the past on testing. In fact, they even tweeted quotes of my own words! I deeply appreciate that someone out there is actually listening, and willing to take the time and effort to hold me accountable to them. I have elected to respond here, since Twitter isn’t the greatest venue for nuanced discussion, especially at the end of a long day, and I also hate typing things on my phone.

This is in reference to a live chat I did back in 2012 on The Nation‘s website with journalist Dana Goldstein and educator Tara Brancato. Have my views shifted since then? I would say they have in some ways.

You know, honestly, they’re not as terrible as I thought back then. I proctor these tests each year and go through the experience of answering the questions along with my students. The questions are often cognitively demanding and require multiple reappraisals of the text in question. A few of them are duds, certainly, but having tried to write many of my own text-dependent questions since then, I’ve come to appreciate a well-written multiple choice question. Check out this post from Joe Kirby (UK educator) on the rationale for using multiple choice questions for assessment.

Unfortunately, this continues to hold true. In reaction to this, the Center for American Progress recently created a “testing bill of rights” to advocate for better aligning tests with a more meaningful purpose.

This doesn’t mean, however, that I’m opposed to having test scores factor into my own evaluation or my school’s evaluation. When scores are considered over multiple years, I think they can be an important and useful measure of teacher effectiveness. But they are extremely variable, so I would only want them to be considered alongside other data that can provide adequate context.

One of the things I’ve become more aware of over time is that while our testing and evaluation schemes are extremely problematic, if we look at the big picture, accountability and testing do bring transparency to serving populations of students that were traditionally ignored. No Child Left Behind was certainly faulty and overzealous policy — but it also brought attention to holding school districts accountable to serving students with disabilities and other underserved populations based on data. This was entirely new, and it has raised awareness.

This is why the NAACP, the National Disability Rights Network, and other national civil rights groups oppose anti-testing movements.

Yes, I continue to believe this. Test measures are only one source of data that need to be coupled with qualitative observational data and other forms of understanding. Fortunately, I do feel like our focus, at least in NYC, has shifted to better match this understanding.

To give further context on my statements on the NYSED video, I was speaking about how I use testing data, which I do every week when developing IEPs for my students with disabilities. I compile all information I have on a student, including multiple years of state test data, in-house assessment data, such as reading, writing, and math scores, GPA, attendance, psychoeducational evaluations, social histories, etc. When viewed all together, in tandem with teacher observations and student and parent interviews, I find aggregate state testing data useful!

So it’s important to understand I’m not advocating now and never have advocated for a state test score as a singular reference point to judge myself or a student. But when viewed with appropriate context, I do find state testing data to be useful. (More on how I use that to develop IEPs here.)

No, unfortunately. While I do think that test scores should factor into an account of an individual teacher’s effectiveness (only in aggregate and when considered in terms of growth, not proficiency), we’re creating incentives for competition, rather than collaboration.

If I could set the rules for how we use test scores for accountability, I would do something kind of radical: I would hold all grade-level teachers accountable for student scores on literacy tests. And I’d stop labeling them “ELA” tests and call them “literacy” tests. Why? Because if we are honest about what we’re really testing, we’d acknowledge that the knowledge required to understand complex texts comes not solely from ELA, but furthermore from science, social studies, music, art, and so forth. (More on my argument on this here).

Furthermore, I’d try to better level the playing field for all students by requiring test makers to broadcast one year in advance which texts would be tested (not specific passages, just the general title/author). I would allow parents and educators an opportunity to vote on which texts they wanted tested that year as well to make it more reflective of current interests. The reason I would do this is that this would provide an opportunity for all students to build up the requisite vocabulary and background knowledge to access a text. Right now we just give them random texts, as if every child will be bringing equivalent knowledge and vocabulary to them, which is false.

Yes, unfortunately this continues to hold true in too many schools. But this is also why I have been a consistent supporter of Common Core standards, which have become synonymous with testing in some people’s minds. Yet the Common Core standards provided us an opportunity to move away from test prep, because they are fundamentally about building student knowledge and academic vocabulary through engagement with rich and complex texts — this is the exact opposite of test prep!

This speaks to the problem of making state tests so high stakes, and why we need multiple measures, such as direct observation, to hold schools accountable. It also is the reason for why I would advocate for the seemingly radical measure, as per above, of communicating which texts would be assessed that year so that “test prep” instead would simply be about reading and studying and discussing the rich texts that were selected for that year’s assessment.

Yes, it can be inhumane when a student is several years behind in reading ability or struggles in coping with anxiety and stress.

While computerized testing brings a whole new set of problems, I do believe we should move in this direction, because with computerized testing, we can use adaptive testing that can better scale to meet a student where they are. Otherwise we end up punishing students who are struggling, for whatever reason. Unfortunately, the needs of students with disabilities never seem to be factored into test design except as a final consideration, rather than from the ground up.

But there’s another side to this, too. I think we have to ask ourselves, as a teacher, a school, and a system, how do we prepare all of our students to be able to engage with a challenging text independently? And in what ways are we sequentially building their knowledge and skills and vocabulary in order to prepare them for doing so? It is the failure to do so systematically and adequately that we are failing students who most need those skills and knowledge.

Pearson is out of the picture, in case you didn’t know. I have no idea what Questar tests will be like, though I imagine they will be comparable.

From what I’ve heard, PARCC assessments are far superior to the cheaper assessments NY decided to get from Pearson. I think we get what we pay for, and if we want better test design, we have to be willing to fund them.

Personally, I think if we’re going to just use tests for accountability purposes, then we could make them every 2 or 3 years instead of every year to save money, and they could still continue to be used for that purpose.

What would be awesome is if we could move more towards performance based assessment. There’s a great article on them in the most recent American Educator. This seems like the right direction to go in if we truly interested in assessing the “whole child.”

Well, don’t know if all of this fully says everything I would like to say about testing, but I’m seriously tired after a long week, so this will have to do.

WiffleCardenal, whoever you are, thank you holding me accountable and I welcome continued critical dialogue on these issues.

* This was after a long day of a train ride from NYC and meetings with legislators, so I apologize for my shiny face. Won’t apologize for the winter beard, however. And no, I was not paid for that interview nor given a script. As ever, I speak my own mind (or so I like to think. Certainly let me know if it ever seems like I don’t).

Cuomo: Build Relationships, Not One-Measure-to-Rule-Them-All

By Codking (Own work) [Public domain], via Wikimedia Commons
In my last post, I explained why NY Governor Cuomo’s push for greater weight on testing was bad strategy. In a study in the American Journal of Public Health on schools with Zero Tolerance policies, here’s some further confirmation that linear thinking backfires in complex environments:

The authors found that “students attending schools with suspension policies for illicit drug use were 1.6 times more likely than their peers at schools without such policies to use marijuana in the next year.” That result held for the student body as a whole — not just for kids who were suspended.

There was another interesting result discovered in this study: relationships are better medicine.

But they did find one factor that actually did seem to decrease the likelihood of drug use: student-teacher interactions. At some schools, students caught with drugs were sent to their teachers to discuss the consequences of drug use. This was associated with a 50 percent decrease in the odds of later marijuana use.

What’s the lesson here for state policy on education?

Don’t seek to impose brute force policies to cover up poor politics and inadequate funding and the most segregated schools in the country. What will really make a difference in education? How about building meaningful relationships with practitioners and the folks on the ground who work in communities and schools facing the greatest of challenges? How about providing them with real funding that demonstrates that we value them?

In the face of complexity, policies that seek to foster empathy and community will be the ones most likely to sustain change.

Source for quotes in this post:

Christopher Ingraham, “Zero-tolerance school drug policies only make drug use worse, a study finds” on The Washington Post, via link on Twitter from RealClearEducation.

School Infrastructure and Student Outcomes

Back in February, Will wrote about “constructing, safer, healthier, more comfortable physical environments for our children” and pointed out that doing so would have an economic benefit. He noted that the Economic Policy Institute has a project to promote this, known as FAST. As I will try to convince you in this post and in subsequent posts, we should be doing everything we can to advocate for FAST.

As I poked around a little bit since yesterday’s post (remember when we used to call it “web surfing?”), I discovered a veritable treasure trove of relevant papers on the topic of learning environments on OECD library‘s CELE Exchange, Centre for Learning Environments. I also found an interesting paper from 2001 on ERIC entitled Building Better Outcomes: The Impact of School Infrastructure on Student Outcomes and Behaviour by Kenn Fisher. Finally, my mom (thanks Mom!) also forwarded me a link to a roundup of studies on this topic from 2000-2009.

In Building Better Outcomes1, Fisher notes that research indicates that 1) “student academic achievement improves with improved building condition”; 2) “individual factors, such as lighting levels, air quality and temperature and acoustics, have an effect on student behaviour and outcomes, although there is limited quantitative evidence available on some of these factors”; and 3) “new and emerging trends in school building planning and design and their impact on student outcomes and behaviour have yet to be evaluated using rigorous research methodology.”

In other words, as of 2001, there was not a lot of evidence of specific impacts of certain environmental factors, but there were obvious trends and academic outcomes and behavior were clearly attached to building conditions. The paper goes deeper into details on factors such as structural and cosmetic factors, color use, the impact of school age (uh oh, think my students are outta luck on that one), and furniture, so go ahead and check it out to learn more.

However, in this review of studies between 2000-2009 published by the 21st Century School FundResearch on the Impact of School Facilities on Students and Teachers: A Summary of Studies Published Since 20002, it is clear that studies point to a meaningful impact of physical school environments on teacher retention and on student outcomes, from attendance and drop-out rates to test scores.

In another paper from 2010 I found on the OECD Centre for Learning Environments collection by Katrien Cuyvers, Well-Being at School: Does Infrastructure Matter?3, they analyzed data on Flemish schools looking for the impact of the physical learning environment on non-cognitive outcomes. In answer to the question of whether differences in student well-being can be attributed to the quality of school infrastructure, it appears clear that there is a connection. There also did not appear to be any significant difference in scores of well-being in rural and urban schools. They noted that “a school with poor quality infrastructure will affect well-being in the same way, whether it be rural or urban.”

It seems clear from the research that has been done that there is a strong link between school infrastructure and student outcomes. So why are we not talking about this in education reform more? Seems to me like this should be front and center — especially when this is a great opportunity to boost our struggling economy.

1. [Fisher, K. (2001), “Building Better Outcomes: The Impact of School Infrastructure on Student Outcomes and Behaviour. Schooling Issues Digest.]
2. [21st Century School Fund. (2009), “Research on the Impact of School Facilities on Students and Teachers: A Summary of Studies Published Since 2000” http://http://www.21csf.org/csf-home/Documents/ResearchImpactSchoolFacilitiesFeb2010.pdf%5D
3. [Cuyvers, K. et al. (2011), “Well-Being at School: Does Infrastructure Matter?”, CELE Exchange, Centre for Effective Learning Environments, 2011/10, OECD Publishing. http://dx.doi.org/10.1787/5kg0lkzc81vc-en%5D

Quote of the Day: The Mismeasure of Man

Stephen Jay Gould’s The Mismeasure of Man is one of the best books available on the use and misuse of data. Given the obsessional focus on test data in education reform these days, I thought it would be a good time to revisit Gould’s work. A few pages into the introduction, I found this wonderful quote:

“We pass through this world but once. Few tragedies can be more extensive than the stunting of life, few injustices deeper than the denial of an opportunity to strive or even to hope, by a limit imposed from without, but falsely identified as lying within.”

Anyone who has learned or taught in a school where test preparation dominates the curriculum has witnessed the stunting Gould writes about firsthand. Those who deny our students the opportunity to study music, art, foreign languages, and evolution deny them the opportunities to strive and hope. That these reformers claim this stunting is for the benefit of our students makes their crime all the more heinous. It recalls the reasoning of biological determinists (like those described and discredited in Gould’s book) who sought to use data to justify their own positions of social and political power.

Arne Duncan Might ‘Get It,’ But Do States?

Arne Duncan recently gave a speech in which he outlined a few targets of education reform that neatly align with our model of schools as ecosystems.

He spoke of the critical need for “wraparound” services, which — as any educator who has worked in a high needs school can attest — is essential. How can children learn when they are hungry, sick, or require glasses?

However, as The Crimson notes, “Duncan said that support services are not enough to bridge the achievement gap and that the quality of schools themselves must be improved.” This is a critical point, but one which needs to be much elucidated. How do we improve the quality of schools? What do we mean by quality?


If we view schools as ecosystems, the answers to these questions become clearer. We improve the quality of schools by fostering relationships and a culture of trust and respect. Quality schools are vibrant communities, rich with interconnections and opportunities and niches for learning.


In his speech, Duncan also succinctly pointed out a key facet in building such environments within schools: “Teacher evaluation should never, ever be based on test scores.”


Of course not. Because if we understand that schools are ecosystems, as opposed to knowledge manufacturing facilities, we know that evaluating children as products is detrimental to the development of a positive, sustainable school community that retains committed teachers and involves and engages parents.

That an influential figure such as Duncan is even talking in this way is a sign of hope. However, as one astute teacher, @KellyDillon1, tweeted in response:

If we’re not to eval teachers based on tests, why does [Race to the Top] push test-based accountability? #mixedsignals

Let’s see if Duncan can put his money where his mouth is and pressure states to focus on the more important necessity for positive school environments and the wraparound community support that he is espousing. And let’s help him deepen his rhetoric with an ecological lens that looks to sustainable growth for every school.