Smorgasbord: NY State Test Results, Incoherency, and Teacher Shortages

NY State test results have been released: trends are positive

This year’s tests can actually be compared directly to last year’s, so inferences are slightly more valid. Statewide, ELA proficiency went up 1.9 points and math 1.1.

It will be interesting to see what narratives spring out of this. Even more interesting will be how anti-charter constituents spin the positive results from charters.

Look for all sides spinning these results in the way that suits them best.

State Education Department Releases Spring 2017 Grades 3-8 ELA and Math Assessment Results, NYSED

Speaking of measurement: How can we measure SEL?

Some interesting suggestions here from a recent design challenge:

  1. How quickly kids answer questions on an on-line test (too quickly means less self-control/engagement)
  2. Asking kids questions about a video to assess their perspective-taking abilities

Building a Modern Marshmallow Test: New Ways to Measure Social-Emotional Learning, EdWeek

It should go without saying that laptops alone do not a quality education make

You know, like, how are you actually using the laptops?

Do Laptops Help Learning? A Look At The Only Statewide School Laptop Program, NPR Ed

How we teach history depends on where we teach it

I’ve argued before that one of the biggest problems with what we teach students across our nation is that it’s completely incoherent, and we do little to nurture a collective sense of values, knowledge, and civic engagement.

Here’s that problem in action:

Virginia’s standards of learning for U.S. history to 1865 include “describing the cultural, economic and constitutional issues that divided the nation” and “explaining how the issues of states’ rights and slavery increased sectional tensions.” Alabama fifth-graders “identify causes of the Civil War from the Northern and Southern viewpoints.”

Contrast that with Delaware, where school districts set their own curriculum but a syllabus for the eighth grade suggesting what might be covered during instruction says that abolition meant that the American people could for the first time “seriously claim to be living up to their commitment to the principle of liberty rooted in the American state papers.”

In Michigan, curriculum also is decided locally, though the state’s social studies standards for the Civil War and Reconstruction in eighth grade include the instructions: “Explain the reasons (political, economic, and social) why Southern states seceded and explain the differences in the timing of secession in the Upper and Lower South.”

Civil War lessons often depend on where the classroom is, Associated Press

Teacher shortages in high needs areas, such as SPED and math, with no end in sight

One of the suggestions here for addressing this makes a lot of sense to me:

“Make teacher certification national instead of state by state. Prospective teachers must pass an exam specific to the state they want to work in. But if a teacher wants to move from, say, Pennsylvania to California, they can’t immediately apply for jobs there. By having a national certification exam, teachers would have more mobility to go where they’re needed.”

Schools throughout the country are grappling with teacher shortage, data show, CNN

One way of addressing teacher shortages in SPED: draw from the paraprofessionals

They’re already in the field. Make it easier for them to transition into teaching.

Makes sense to me. But one thing to be aware of: paras have great experience in managing behaviors and working with kids, but may not have a strong background on content.

Which is why having a strong curriculum and departmental teams that can support adaptation and implementation of that curriculum are so critical.

With principals in ‘crisis mode,’ new Washington state law taps into thousands of potential teacher recruits, Seattle Times

Advertisements

Success can’t be measured by one or two numbers

“Whenever you make huge decisions about complex situations based on one or two numbers, you’re headed for disaster — especially when those numbers can be gamed.”

—Mark Palko and Andrew Gelman, “How schools that obsess about standardized tests ruin them as measures of success” on Vox

We’ve questioned Success Academy’s “success” on this blog before. These statisticians bring a new lens to that question.

I don’t want to denigrate the good work that Success Academy teachers and students are doing. There are practices and systems well worth replicating and investigating in these schools. But Eva Moskowitz’s political framing and marketing of her schools as the solution to poverty is problematic.

College and Career Ready? Maybe neither

Last week, I wrote about how NY was moving to lower high school diploma expectations for students with disabilities. Since writing that post, the NY Board of Regents has voted in the law, effective immediately, which has created some confusion for principals.

I’ll admit I know little of the landscape of NY high school exit requirements, since I’ve spent my career at the elementary and middle school levels. What remains unclear to me is what a “local diploma” really means, and how it connects to a viable career, as some advocates for students with disabilities are saying (as reported in this Chalkbeat piece). I’m open to being further educated on this, if anyone out there wants to school me. But right now it seems to be a mechanism for diminished expectations for some students, while enabling adults to claim higher grad rates.

Chalkbeat reporters Alex Zimmerman and Annie Ma further report that “Todd Kaminsky, a state senator who pushed for the new graduation requirements, said the change isn’t about watering down standards, but paving the way for more appropriate, “project-based” measures for students who struggle to meet graduation requirements.”

It’s also unclear to me how reducing requirements for students with disabilities connects to “project-based” measures, as this is not an explicit component of the law itself, which you can view in an overview of on this document provided by NYSED. I’m all for performance-based assessment (which is maybe what Kaminsky meant to refer to—to my knowledge, project-based learning is a pedagogical strategy, not a form of assessment), but utilizing PBA does not require lowering expectations. If these supplanted the traditional Regents exams, I’d be all for it. But I still wouldn’t stand by reducing expectations for students with disabilities.

On Twitter, The74’s Matt Barnum challenged my thinking on high school diploma requirements:

His post provides an overview of research which suggests that stringent high school diploma requirements may have little of the expected benefits (increased academic achievement), while it can have many unintended downsides, such as an increase in drop-out and incarceration rates.

I find this research compelling and a fit rebuttal to the imposition of high standards without compensatory attention paid to providing alternative options.

But I still don’t think lowering expectations for an academic diploma for some, or any, students is the answer. A high school diploma should signify that a student is prepared to enter college.

Not all students are prepared to enter college, whether due to ability or interest. However, all students could be better equipped to begin a career.

The greatest underreported story of last year, in my opinion, is that dramatically greater numbers of students are now failing the GED. This is far more problematic than students failing to obtain a HS diploma.

Couple this with the general dearth of well designed and funded vocational programs and opportunities in the US.

Over in Kentucky, however, there is a more sane and equitable approach that does not require diminishing expectations, as Emmanuel Felton reports. In KY, they are building two tracks between what it might mean to be “college” and/or “career” ready, and this makes a lot of sense to me. Instead of devaluing a high school diploma just to allow states to claim higher graduation rates, we should be investing in alternative pathways to a career that are both viable and rigorous.

 

My current views on testing, in answer to my past views on testing

While up in Albany a few weeks ago, I was interviewed by someone from NYSED about what I might say to parents who are considering “opting out” their child from state testing. You can view the video here*.

Someone on Twitter, “WiffleCardenal,” voiced a critique to me regarding the video, in contrast to things I’ve said in the past on testing. In fact, they even tweeted quotes of my own words! I deeply appreciate that someone out there is actually listening, and willing to take the time and effort to hold me accountable to them. I have elected to respond here, since Twitter isn’t the greatest venue for nuanced discussion, especially at the end of a long day, and I also hate typing things on my phone.

This is in reference to a live chat I did back in 2012 on The Nation‘s website with journalist Dana Goldstein and educator Tara Brancato. Have my views shifted since then? I would say they have in some ways.

You know, honestly, they’re not as terrible as I thought back then. I proctor these tests each year and go through the experience of answering the questions along with my students. The questions are often cognitively demanding and require multiple reappraisals of the text in question. A few of them are duds, certainly, but having tried to write many of my own text-dependent questions since then, I’ve come to appreciate a well-written multiple choice question. Check out this post from Joe Kirby (UK educator) on the rationale for using multiple choice questions for assessment.

Unfortunately, this continues to hold true. In reaction to this, the Center for American Progress recently created a “testing bill of rights” to advocate for better aligning tests with a more meaningful purpose.

This doesn’t mean, however, that I’m opposed to having test scores factor into my own evaluation or my school’s evaluation. When scores are considered over multiple years, I think they can be an important and useful measure of teacher effectiveness. But they are extremely variable, so I would only want them to be considered alongside other data that can provide adequate context.

One of the things I’ve become more aware of over time is that while our testing and evaluation schemes are extremely problematic, if we look at the big picture, accountability and testing do bring transparency to serving populations of students that were traditionally ignored. No Child Left Behind was certainly faulty and overzealous policy — but it also brought attention to holding school districts accountable to serving students with disabilities and other underserved populations based on data. This was entirely new, and it has raised awareness.

This is why the NAACP, the National Disability Rights Network, and other national civil rights groups oppose anti-testing movements.

Yes, I continue to believe this. Test measures are only one source of data that need to be coupled with qualitative observational data and other forms of understanding. Fortunately, I do feel like our focus, at least in NYC, has shifted to better match this understanding.

To give further context on my statements on the NYSED video, I was speaking about how I use testing data, which I do every week when developing IEPs for my students with disabilities. I compile all information I have on a student, including multiple years of state test data, in-house assessment data, such as reading, writing, and math scores, GPA, attendance, psychoeducational evaluations, social histories, etc. When viewed all together, in tandem with teacher observations and student and parent interviews, I find aggregate state testing data useful!

So it’s important to understand I’m not advocating now and never have advocated for a state test score as a singular reference point to judge myself or a student. But when viewed with appropriate context, I do find state testing data to be useful. (More on how I use that to develop IEPs here.)

No, unfortunately. While I do think that test scores should factor into an account of an individual teacher’s effectiveness (only in aggregate and when considered in terms of growth, not proficiency), we’re creating incentives for competition, rather than collaboration.

If I could set the rules for how we use test scores for accountability, I would do something kind of radical: I would hold all grade-level teachers accountable for student scores on literacy tests. And I’d stop labeling them “ELA” tests and call them “literacy” tests. Why? Because if we are honest about what we’re really testing, we’d acknowledge that the knowledge required to understand complex texts comes not solely from ELA, but furthermore from science, social studies, music, art, and so forth. (More on my argument on this here).

Furthermore, I’d try to better level the playing field for all students by requiring test makers to broadcast one year in advance which texts would be tested (not specific passages, just the general title/author). I would allow parents and educators an opportunity to vote on which texts they wanted tested that year as well to make it more reflective of current interests. The reason I would do this is that this would provide an opportunity for all students to build up the requisite vocabulary and background knowledge to access a text. Right now we just give them random texts, as if every child will be bringing equivalent knowledge and vocabulary to them, which is false.

Yes, unfortunately this continues to hold true in too many schools. But this is also why I have been a consistent supporter of Common Core standards, which have become synonymous with testing in some people’s minds. Yet the Common Core standards provided us an opportunity to move away from test prep, because they are fundamentally about building student knowledge and academic vocabulary through engagement with rich and complex texts — this is the exact opposite of test prep!

This speaks to the problem of making state tests so high stakes, and why we need multiple measures, such as direct observation, to hold schools accountable. It also is the reason for why I would advocate for the seemingly radical measure, as per above, of communicating which texts would be assessed that year so that “test prep” instead would simply be about reading and studying and discussing the rich texts that were selected for that year’s assessment.

Yes, it can be inhumane when a student is several years behind in reading ability or struggles in coping with anxiety and stress.

While computerized testing brings a whole new set of problems, I do believe we should move in this direction, because with computerized testing, we can use adaptive testing that can better scale to meet a student where they are. Otherwise we end up punishing students who are struggling, for whatever reason. Unfortunately, the needs of students with disabilities never seem to be factored into test design except as a final consideration, rather than from the ground up.

But there’s another side to this, too. I think we have to ask ourselves, as a teacher, a school, and a system, how do we prepare all of our students to be able to engage with a challenging text independently? And in what ways are we sequentially building their knowledge and skills and vocabulary in order to prepare them for doing so? It is the failure to do so systematically and adequately that we are failing students who most need those skills and knowledge.

Pearson is out of the picture, in case you didn’t know. I have no idea what Questar tests will be like, though I imagine they will be comparable.

From what I’ve heard, PARCC assessments are far superior to the cheaper assessments NY decided to get from Pearson. I think we get what we pay for, and if we want better test design, we have to be willing to fund them.

Personally, I think if we’re going to just use tests for accountability purposes, then we could make them every 2 or 3 years instead of every year to save money, and they could still continue to be used for that purpose.

What would be awesome is if we could move more towards performance based assessment. There’s a great article on them in the most recent American Educator. This seems like the right direction to go in if we truly interested in assessing the “whole child.”

Well, don’t know if all of this fully says everything I would like to say about testing, but I’m seriously tired after a long week, so this will have to do.

WiffleCardenal, whoever you are, thank you holding me accountable and I welcome continued critical dialogue on these issues.

* This was after a long day of a train ride from NYC and meetings with legislators, so I apologize for my shiny face. Won’t apologize for the winter beard, however. And no, I was not paid for that interview nor given a script. As ever, I speak my own mind (or so I like to think. Certainly let me know if it ever seems like I don’t).

Charter vs. District Systems

By NASA’s Aqua/MODIS satellite (http://visibleearth.nasa.gov/view_rec.php?id=6204) [Public domain], via Wikimedia Commons
Neerav Kingsland looks at the recent findings on professional development via the TNTP Mirage report and the Rand Corporation study, and comes to the conclusion that “Professional development only seems to lead to student achievement increases in charter schools!”

I noted in a recent post that in the TNTP study, teacher effectiveness and growth was notably more observable in a CMO, and I hypothesized that this could be attributable to some charter networks having more tightly managed systems of assessment, curriculum, teacher practice, and observation.

But to suggest that this is an innate quality of charter schools is questionable. There is absolutely no reason for a district school not to be in possession of such qualities, and indeed, many do.

Kingsland argues for NOLA-style systems, in which the government merely regulates, rather than operates, schools, with the idea being that the private sector can conduct operations more efficiently and effectively. But there’s a potential, and possibly critical, issue with such a system: a lack of coherency.

Within a well-managed district, on the other hand, there is potential for greater coherency. A state or central office can provide specific direction on operational shifts via policy that all district schools would be expected to adhere to.

Kingsland asks, “is it more likely that we can achieve major gains in districts or scale highly effective charters?,” I think he’s created a false dichotomy. I think the more interesting question is, “How can we achieve major gains by leveraging federal, state, and district policy to implement effective and coherent systems, content, and practices across all schools?”

A NOLA-style system might be able to make swift initial gains, due to well-managed networks putting into place strong systems of assessment, feedback, and practice. But it’s certainly feasible that a well-managed district system can make even bigger gains over the longer haul.

I disagree, therefore, with Kingsland’s position that charter schools are inherently superior in enhancing teacher effectiveness and promoting student achievement. In fact, I charge that a NOLA-style system may ultimately run up against its innate incoherency, at which point, large-scale gains would stagnate.

I could be totally wrong on this, of course, and admit that this is conjecture and based on my own values. It may be that a NOLA-style system may end up leading to greater coherency in operations due to competition, and thus, best practices evolve through demonstrated gains in one organization and subsequent adoption by those who are attempting to compete.

I may also be overstating the ability of district schools to establish coherency, given constraints in operating within often volatile political contexts.

The problem is, of course, that while NOLA has demonstrated significant academic gains on tests since moving into a private sector operated system, it’s still purely conjecture as to whether the same benefit would transfer to any other district simply due to a  structural change. It’s also still conjecture that those gains can be solely attributed to a structural shift to private sector operation, rather than the simple mechanism of distributing students across geographical boundaries.

But let’s assume for the moment that Kingsland is correct that a private sector operated school system is the optimal system. I would still argue, even in such a case, that this doesn’t mean that such a system will necessarily scale effectively into different social and political contexts.

In the face of great complexity and uncertainty, we can hedge our bets by planning for robustness, rather than optimality.

The question therefore becomes: what is the most robust? A school system operated by the public, or a school system operated by the private sector?

Perhaps the answer lies somewhere in between.

Why the new ESEA won’t change US education

12de7-statescolor

I haven’t been following the ESEA revisions closely as they’ve meandered their way through the Senate and the House, but my general impression is that this new bill will not dramatically transform public education in the United States (surprise!).

My understanding is that revisions as they currently stand are looking to:

The unfortunate fact is that perennial political debates between federal and state rights assumes precedence over the practical exigencies of public education. Transforming an entire nation’s education system requires strong federal guidance and authority.  The most current iterations of ESEA suggests that federal authority will be yet further curtailed.

This isn’t to say that NCLB was perfect. But the federal government clearly established higher standards and accountability to those standards for all students. This was critical for students that had been historically denied access to a quality education, such as students with disabilities. Yet NCLB was admittedly inflexible and created the unintended consequence of a single-minded focus on superficial ELA and math tests that were largely disconnected from any viable curriculum.

As problematic as the testing regime has been, however, tests could serve a more proactive instructional purpose if they were better designed to acknowledge the importance of content knowledge in literacy, if they were better able to adapt to student ability, and if they incorporated community feedback in the selection of the texts they assessed*. Then tests could actually provide direction on curriculum and instructional decisions. I’ve made this argument before, and so has David Steiner, former education commissioner of NY State, just so you know I’m not crazy.

I appreciate that senators are pushing for the creation of “an evidence-based innovation fund,” but I fear that such funds most likely won’t be put towards building infrastructure, despite a growing body of research demonstrating the impact of environment on behavior and learning.

I also greatly doubt that ESEA will be looking to fund or create accountability for any efforts at integration of schools or communities based on socio-economic status, despite evidence demonstrating its impact on reducing the opportunity gap.

I applaud our public representatives for working together across the aisle to negotiate these revisions of NCLB. But I am afraid that their efforts will do little to better outcomes for the majority of our students. Our nation needs to get real about increasing federal authority over curriculum, funding, and regulating school choice.

Let me be clear that I don’t mean that I’m advocating for one-size-fits-all. I believe states and districts require greater flexibility and choice in adapting funding and policies to their students’ needs. But without strong guidance and accountability from the federal government, states will be highly unlikely to do the heavy moral and financial lifting of pushing for more integrated schools and neighborhoods, designing or adopting tests that are adaptive yet more rigorous and content-rich, creating new school buildings that increase access to natural light and greenery, or pushing for increased access to a quality school and program and teacher for traditionally ignored populations.

*Update: I should add that tests should further be tied to rigorous, common standards.

Cuomo Doubles Down on Linear Thinking

“The dunes here are linear, thought to be due to shifting wind directions.”

New York’s Governor, Andrew Cuomo, has decided to double down on a misguided effort to increase the weight of state assessments in a teacher’s evaluation from 20% to 50%.

I’m going to spare you the “corporate reform” and “hedge fund buddies” angle on this; instead, I contend that this single-minded focus on test scores is simply bad strategy.

The focus of this blog has been on teasing out the metaphor of a school as an ecosystem, and elaborating on the theme that managing complex systems requires moving beyond linear thinking.

Here’s a relevant quote from Steve Denning (which we’ve examined before) on the principle of obliquity:

Efforts to impose linear thinking on complex situations have often led have the opposite of what was intended. As a result, the principle of obliquity becomes relevant. Where explicit articulation of a goal will result in the complex environment pushing back in the opposite direction, oblique goals will often be more effective, e.g. the goal of delighting customers may make more money than an explicit goal of making money.

What would be the opposite of what was intended in this situation (if Cuomo gets his way)? Well, if every teacher in the state is conscious that their evaluation is heavily determined by their student’s performance on that state test — then the problems of focusing mostly on ELA and math and shallow skills-driven test-prep will most likely be exacerbated. And kids that most need access to rich literature and knowledge across the domains of history, music, arts, science, and technology will instead continue to be given drivel. And teachers and schools may be more likely to engage in cheating.

I generally assume best intent when assessing the decisions of others. So to be fair to Cuomo, he is pushing for a simplified accountability system because the current system of 20% state, 20% local, and 60% principal observation may lead to the problems of over-testing and inflated scores.

But moving to increase the weight of the state test scores as a leverage over teachers is not the right move to resolve these issues. Instead, this maneuver is much more likely to compound deeper issues, rather than achieve the goal of increasing student and teacher performance.

It would be great if our elected representatives could move beyond linear models when making critical decisions on how to improve our system of education.

Further Thoughts on Testing

Last Thursday I participated in a live chat on standardized testing and its role in education reform. You can view the transcript of the discussion online.

Here are some of the main points I made about testing during the discussion:

  • Multiple choice tests largely measure shallow skills, not higher order thinking
  • Attempts to measure deeper thinking via “efficient” measures is quixotic
  • Isolates skills from real content – no connection to actual curriculum taught
  • Results in test prep curriculum – shallow skills with little relevance to meaningful texts
  • Test prep curriculum removes engagement with meaningful texts from those students who are most in need of access to rich literature
  • Used primarily to evaluate teachers, not to diagnose students
  • Based on proficiency
  • Needs of students with disabilities are not factored into test design – accommodations are an afterthought
  • Causes suffering to students who are struggling
A NYC public school teacher, Claire Needell Hollander, wrote a great article on the NY Times entitled Teach the Books, Touch the Heart. She develops the argument that current tests are devoid of real literature, and thus test prep similarly removes great literature and the emotional connection it provides from the students who most need access to this form of “cultural capital.”

We should abandon altogether the multiple-choice tests, which are in vogue not because they are an effective tool for judging teachers or students but because they are an efficient means of producing data. Instead, we should move toward extensive written exams, in which students could grapple with literary passages and books they have read in class, along with assessments of students’ reports and projects from throughout the year.

This is good advice. By connecting tests to the actual curriculum taught, we can avoid the tunnel vision of test prep.

Another great article on testing was recently posted on Washington Monthly, by Ed Sector‘s Susan Headden, entitled A Test Worth Teaching To. Headden notes many of the same issues that Hollander points out, and she also points out that tests are designed to be efficient and cheap, and thus don’t measure the higher order thinking that open ended questions would promote.

Headden is hopeful that the new tests designed by the Common Core testing consortiums will be tests worth teaching to, because they will be more akin to the open ended, higher order thinking challenges posed by AP and IB tests. She also notes that they will be computerized and adaptive, with performance learning tasks that can better diagnose students’ deeper analysis capabilities.

I am also hopeful about the new tests and believe that the adaptive nature of the questions will provide much more timely and useful information. However, I continue to remain skeptical of whether a test that assesses skills isolated from the actual curriculum taught can really be a great improvement.

There might be one other non-robotic way to bring down the cost of scoring: assign the task to local teachers instead of test-company employees. According to the Stanford Center for Opportunity Policy in Education, the very act of scoring a high-quality assessment provides teachers with rich opportunities for learning about their students’ abilities and about how to adjust instruction. So teachers could score assessments as part of their professional development—in which case their services would come “free.”

This is also good advice. I think scoring these deeper tests via local teachers would provide a great learning opportunity for the teachers to get deep into the questions and understand where their students are struggling.

So if we put Headden’s and Hollander’s advice together, we could perhaps have a test worth teaching to: tests based on real literature that students have read during the year, scored by local teachers instead of test-company employees or computers. Then if we also consider the needs of students with disabilities from the outset of test design, rather than as an afterthought, we could truly have some great tests.

But as I said on the live chat, we also need to stop our obsession of using tests as evaluative instruments. We could move testing to a randomized or staggered basis (every 2 or 3 years) and put the remaining money to the much more important direct observation of school learning environments and assessments of school curricula.

 Efficient? No. But well worth the undertaking, given the issues outlined in our live chat.

It Takes More Than War PART II: A Critique of Geoffrey Canada’s Approach to Ed Reform

This is the second post in a series critiquing Geoffrey Canada’s approach to education reform. Read Part I.

In my last post examining Paul Tough’s account of Geoffrey Canada’s initiation of the Harlem Children’s Zone (HCZ), I noted that Canada brought much needed focus to “wraparound services” and early childhood years in addressing the achievement gap. However, I critiqued Canada’s approach by pointing out that despite his own acknowledgement of the importance of a long-term perspective, he focused on short-term, bottom-line indicators, putting great pressure on his administrators, teachers, and students to the detriment of an inclusive learning environment that supports and nurtures social and emotional needs.

In this post, I’d like to continue my critique of this bottom-line approach, but before I do so, I want to make it clear that I am not attempting to devalue Geoffrey Canada’s work. He is a pioneer, and he has established clear pathways to raising Harlem’s children out of poverty. I am attempting to build on this work by promoting a deeper focus on creating inclusive, positive learning environments that address the whole child as the next step in promoting education reform.

Something else I appreciate about Canada is his utter refusal to give up on poor black communities, and rather than try to isolate children from the culture they were raised within, he seeks to transform the community as a whole through what he calls the process of “contamination.”

Many programs that try to help poor children, including charter schools, charities, and social service agencies, take as their premise that the best way to help children in a bad environment is to separate them as much as possible from that environment, to insulate them from the problems and values of the ghetto or even to extract promising kids from their homes and drop them into elite boarding schools. Canada, by contrast, wants to leave Harlem’s poor children exactly where they are, so that they change the neighborhood and the neighborhood changes them.

In order to give students the skills that they were sorely lacking, Canada’s school team focused on test results, just as so many public schools are prodded to do today in the current “data-driven” environment. This focus is due in large part to research on poverty by James Heckman:

Skills matter. The more ability you have, the better you are likely to do in life. . . . significant skill gaps exist–by race, class, and maternal education–and they open up very early.

Recognition of these gaps in skills has therefore resulted in what we call “test prep,” in which kids are drilled on skills like finding the meaning of a word using context clues, figuring out the main idea of a text, or making an inference.

But focusing primarily on reading skills in this way does a disservice to students who are struggling with those basic skills in two ways. First of all, it ignores the social and emotional needs that all students have. Students that have been raised in conditions of poverty often have great emotional and psychological needs that are direly evident to those who work with them. They need a lot of attention, empathy, and love. As Tough puts it in his narrative:

It was the X factor, the magic ingredient that could outweigh all the careful calculations behind Promise Academy’s strategy for success: on top of the hours and hours of cognitive training, what made the difference in many students’ lives was a personal connection that was impossible to measure and difficult to replicate. If the kids didn’t get that, all the tutoring in the world might not help them.

The second way that test prep does a great disservice to kids is that the reason they are struggling with basic skills in literacy is not because they need to be taught reading skills, but because they lack the background knowledge essential to comprehending and grappling with complex texts. E.D. Hirsch, Jr., Daniel T. Willingham, and Robert Pondiscio have argued this point more in full elsewhere, but to sum up a major point they have all made: reading is not really a skill, it is more a reflection of prior knowledge. Willingham puts it well here:

We tend to teach comprehension as a series of “reading strategies” that can be practiced and mastered. Unfortunately it really doesn’t work that way. The mainspring of comprehension is prior knowledge—the stuff readers already know that enables them to create understanding as they read.

So what is it that kids need, then? As I have argued on GothamSchools, they need rich, coherent curriculum that targets their social and emotional needs, in addition to building up their background knowledge with structured, sequential content.

Yet this is almost the exact opposite tack that Canada’s principal of his elementary school took:

McKesey’s belief was that on tests like the one the 2nd grade students were practicing to take, knowledge was only one part of the equation, and a fairly small part, at that. It wasn’t that he didn’t believe in teaching content . . . it was just that he thought that when a child sat down to take a standardized test, knowing a lot of information wasn’t necessarily all that helpful. What mattered more was having mastered certain test-taking strategies and tricks, the kind that allowed students to undertand what the exam was really asking.

Wrong. Knowing a lot of information is extremely helpful. Knowing a lot of strategies will maybe get the school some short term boosts on tests, but it’s not going to take those children anywhere. This kind of approach does an extreme disservice to children, especially ones who are already severely behind. They need more knowledge, more deep understanding of the academic fields they are expected to master, not shallow strategies on how to succeed on shallow multiple choice tests.

Furthermore, when children are suffering with great emotional or psychological needs, creating an environment of high stress and emotional vacuity is detrimental to their long-term growth. Eventually, what happens is that some kids — the ones termed “bad apples” in Tough’s account — will be swept under the rug. They need an environment of structured caring, with content that is meaningful and that will enrich their lives and broaden their perspectives.

It takes so much more than war to save our communities and our children most in need. It takes a lot of love and empathy. It takes a structured, consistent focus on restructuring our economy, redesigning our urban spaces, and measuring the things that truly matter. We must stop focusing on the bottom-line of multiple choice tests, and focus instead on the bottom-line of the hearts and minds and well-being of children.

Pineapple Express: Tests Shortchanging Student Literary Analysis Skills

The infamous Pineapple Passage on the 8th grade NY state test is rightfully making the rounds on  netmedia. It’s a prime example of something that has surprisingly thus far gone relatively unmentioned*, which is that as test-makers attempt to make test questions “higher order” in the form of inference and reading-between-the-lines, they necessarily skirt the fine line between what is easily quantifiable and what must be qualified by interpretation.

I noticed on my 5th graders’ ELA exam this past week that many of the questions were so subject to interpretation as to be perplexing as a multiple choice question.

I was an English major in college. Though I can’t claim to have engaged extensively in it, literary analysis is something I am not a complete stranger to, and I know that literary criticism can be highly subjective (though not as subjective as non-fuzzy major folks may assert). Much like in the art critic world, consensus towards a perspective on a particular work is arrived at via a long-form process of back-and-forth akin to peer review. Papers are written, professors stake careers on counterpoints, and over time, paradigms shift and the critique of a given work merges with the living history of a society. (Not sure if this last part was stated very clearly by the way, but I’m trying to push this post out while the issue is still relevant, so let me know how I can re-write that.)

It’s via the process of dialogue, therefore, that perspectives on literature evolve. It’s qualitative. You can’t assign a number to it without putting it into context.

Yet test-makers, due to pressure from policymakers concerned foremost with the short-term and the political, are attempting to assign numbers to the process of deeper literary analysis, which simply can’t be done. Ostensibly, they are measuring reading comprehension, but this disassociated push for “higher order thinking” mistakes simple comprehension of plot, setting, and character for deeper interpretation of what the text might mean.

For example, on the 5th grade test, there was a story about a family who lives in a cabin, and a blizzard suddenly occurs. Through a misunderstanding, the mother (or grandmother, I don’t remember) gets locked in the cellar by the father, who doesn’t know she’s in there. When he finally realizes what has happened, he opens it up, and the mother comes out, rubbing her hands and stomping her feet, and she quips something along the lines of “If you were going to lock me up somewhere, it should have been in the barn.”

A question then asks (wording might not be exact): After coming out of the cellar, the mother MOST LIKELY felt:
A. amused
B. anxious
C. angry
D. relieved

I was perplexed by this one. As I train my students to do, I kept going back to the passage to re-read the section on when she came out for evidence. She could be argued to have been somewhat amused, because she cracks a joke the moment she pops out. She may possibly be angry. One can infer that she is relieved, because who wouldn’t be relieved to be released after being locked up in the darkness?

The answer they want, obviously, is that she is relieved. But given that one could make an argument, based upon the evidence from her statement and via making a deeper inference about her evident dry wit, that she was also amused, it seems highly suspect to give a kid 0 points for one answer, and 1 point for the other. In other words, if you are testing a kid’s inferencing ability, well, then both answers are plausible when applying that skill.

There were a number of questions like this throughout the test. They are most certainly challenging questions, and interesting from a purely academic standpoint. But they are not amusing to me as I watch my students with exceptional learning needs with their heads bowed for over 2 hours grappling with passages that are well above their reading level. I witness children who whisper “I can’t do this” and put their heads down in the middle of the test. As GothamSchools noted in a recent article, this is akin to torture, and this grand experiment by short-sighted adults who simplemindedly clamor for quick and easy data has real and very human consequences on children.

There is a value and purpose for multiple choice questions. It’s like sticking your finger into the air after licking it. It gives you a quick sense in what direction the wind blows. But we need to stop pretending that we’re getting a true picture of an individual child’s ability to analyze and infer. So while NY may have “canned the pineapple,” we need to can the tests.

*This great opinion piece in the NY Times, Teach the Books, Touch the Heart, makes a parallel argument on how tests are hardly culture neutral, and how we should scrap multiple choice tests altogether and host written exams based on passages and books children have read in class. Great advice, and this is a must-read. Thanks to @KellyDillon1 for tweeting out the link to this.