Reviewed by Jeremy Levine, University of Massachusetts, Amherst
Murphy, S., & O’Neill, P. (2023). Assessing writing to support learning: Turning accountability inside out. Routledge.
Sandra Murphy and Peggy O’Neill’s (2023) Assessing Writing to Support Learning: Turning Accountability Inside Out synthesizes existing research on writing assessment, psychometrics, and writing pedagogy to argue that teachers should be at the center of the school accountability system. Foregrounding formative assessment processes such as portfolio grading, Murphy and O’Neill propose a framework through which ecological writing assessment (which has been applied at the post-secondary level, per Wardle and Roozen 2012; Inoue 2015) can be brought to K-12 instruction. In the book’s first chapter, they argue that such a pivot will reduce the extent to which high-stakes assessment narrows writing curricula, account for a fuller picture of writers’ knowledge aligned with modern research, and include teachers as active decision-makers in writing assessment. This claim illuminates the administrative and policy risks of hitching the K-12 writing assessment wagon to standardized tests: because of their limited view, tests can misguide administrators and the public about what our students know about writing. On the teaching side, the emphasis on the narrowed curriculum could also include examination of the contextual nature of testing’s influence on instruction (McCarthey, 2008) and how teachers mediate testing expectations through their own goals for writing (e.g. Wahleithner, 2018). These local concerns shift the book’s exigence slightly: teachers are already making writing assessment their own; a more productive policy paradigm would build on this teacher agency, rather than create obstacles for it.
Chapter Two is a crash course in writing assessment, overviewing the fundamental concepts of reliability and validity. Validity is of particular interest to Murphy and O’Neill, who make two validity-based critiques of high-stakes testing. The first is that the accountability system must take consequential validity seriously: that the purpose of administering a test affects how teachers and students approach it, meaning the curricular changes that accompany high-stakes testing are a threat to the test’s validity. Second, standardized testing has weaknesses in terms of construct validity: the extent to which a test measures what it claims to. The construct validity critique is built on the concept that student text is not necessarily a stand-in for student writing knowledge, as a student’s ability to produce a specific genre under testing circumstances cannot speak to their rhetorical flexibility or approach to writing across genres, purposes, or settings. This claim about construct validity helpfully builds on the growing body of research that locates substantial portions of writing development as taking place off the page, including concepts such as dispositions and identities (see Driscoll & Zhang, 2022). The importance of each of these concerns is made clear in Chapter Three, which focuses on evolving theories of writing and writing instruction. Accounting for both social and cognitive theories of writing, O’Neill and Murphy offer an overview of writing concepts (e.g. writing as expression, writing as a product, writing as a social activity, etc.) and instructional practices (writing for a real audience, building genre knowledge, participating in peer review, reflecting). Composition researchers will surely recognize these lists of concepts, but they do important work in demonstrating how out-of-step a high-stakes exam is with theories of writing instruction (a blow to its consequential validity) and to how writing is understood (a blow to its construct validity).
With these flaws in high-stakes assessment established, the rest of the book pivots toward solutions. The first of two goals in Chapter Four is to outline classroom-scale models of formative assessment that give students opportunities to reflect on their own writing processes. To illustrate the rigor of such formative assessments, and demonstrate their promise of improving metacognition, O’Neill and Murphy offer several examples of self-assessment rubrics that may help teachers and students identify key facets of the writing process to focus on. The second goal of Chapter 4 is to describe approaches for large-scale assessment practices that align with the cognitive and social characteristics of writing described in the third chapter. This means conceiving of writing as a task- or context-specific activity, which in testing circumstances might involve portfolio assessment, assessments that integrate reading and writing, collaborative writing, and digital or multimodal writing tasks. These recommendations culminate in the authors’ invocation for accountability to be turned “inside-out,” putting the complexity of writing, and the needs of students, at the forefront of writing assessment on a large scale, rather than prioritizing the psychometric approach of standardization and controlling variability. These arguments for rewriting large-scale writing assessment lead to questions about what happens when these measures are attached to accountability systems. For example, portfolio assessment at the state level may absorb all student writing into a bureaucratic system (Scott, 2008), or schools may feel undue pressure to improve, say, “writing motivation” scores, and as a result, could focus more on scores than on actual writing motivation (see Koretz, 2017). O’Neill and Murphy make a strong case for why these elements must be included in large-scale assessment; the question is what the implications might be once that happens.
Chapters Five and Six offer strategies for bringing the recommendations from Chapter Four into reality. Chapter Five focuses on methods for redesigning writing assessment, arguing that teachers need to be at the center of assessment processes because teachers are ultimately responsible for implementing classroom changes. To make this change a reality, O’Neill and Murphy propose investing in professional development, involving teachers in assessment design, and supporting collaboration across levels of education. The sixth and final chapter of the volume focuses on an ecological model of writing assessment. Building on the work of Inoue (2015) and Wardle and Roozen (2012), both of whom focus on assessment ecologies at the post-secondary level, the authors offer an invocation for similar frameworks to make their way into K-12 schools. Combined with the emerging psychometric concept of ecological validity, this chapter’s focus on ecologies creates a “springboard for action” (p. 192) that can mobilize teachers and researchers toward shifting the terms of control for assessment and accountability in the United States. This strategy combines rather nicely with parallel calls for reform in education studies, which suggest student surveys (Schneider et. al, 2021) or inspectorates (Berner, 2017) may be more productive (and ecologically-minded) assessment systems. As O’Neill and Murphy conceptualize it, writing can be a productive window into school life, thereby giving it the potential to be especially useful in these imagined reforms.
In total, Assessing Writing to Support Learning: Turning Accountability Inside Out offers conversation-starting concepts for multiple audiences. For policymakers at school, local, and even state levels, it illustrates how modern conceptions of accountability are out-of-step with best practices in writing instruction and assessment. For instructors at the primary, secondary, and post-secondary levels, it invites reflection around how classroom assessment practices can be used to foster students’ learning about writing, and when or how such assessment practices can become disconnected from assessment. For researchers on writing, it offers a framework for conceptualizing validity on ecological terms and invites future inquiry on the intersection of classroom assessment and policy concerns. Primarily grounded in research and concepts from writing studies, its connections to education reform – implementation, accountability, and possibilities for reform – leave lingering questions regarding how the proposed ecological model of assessment can be implemented as policy. At its core, the text is a reminder that classrooms are about relationships between students and teachers, and this relationship — not the concerns of parties outside of that room — should be at the center of conversations about learning.
References
Berner, A (2017). Would School Inspections Work in the United States? Johns Hopkins School of Education, Institute for Education Policy.
Driscoll, D. L., & Zhang, J. (2022, March). Mapping long-term writing experiences: Operationalizing the writing development model for the study of persons, processes, contexts, and time. In Composition Forum (Vol. 48). Association of Teachers of Advanced Composition.
Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. Parlor Press.
Koretz, D. (2017). The testing charade: Pretending to make schools better. University of Chicago Press
McCarthey, S. J. (2008). The impact of No Child Left Behind on teachers’ writing instruction. Written Communication, 25(4), 462-505
Murphy, S., & O’Neill, P. (2022). Assessing writing to support learning: Turning accountability inside out. Taylor & Francis.
Schneider, J., Noonan, J., White, R. S., Gagnon, D., & Carey, A. (2021). Adding “student voice” to the mix: Perception surveys and state accountability systems. AERA Open, 7, 1-18.
Scott, T. (2008). “Happy to comply”: Writing assessment, fast-capitalism, and the cultural logic of control. The Review of Education, Pedagogy, and Cultural Studies, 30(2), 140-161.
Wahleithner, J. M. (2018). Five portraits of teachers’ experiences teaching writing: Negotiating knowledge, student need, and policy. Teachers College Record, 120(1), 1-60
Wardle, E., & Roozen, K. (2012). Addressing the complexity of writing development: Toward an ecological model of assessment. Assessing Writing, 17(2), 106-119.