JWA Reading List

Greetings!

The Journal of Writing Assessment’s Reading List is thrilled to release our Summer 2024 Issue!

Our reviews in this issue explore two exciting books from the last year that make important contributions to the field of writing studies, and to the area of writing assessment more specifically. These texts have relevance for a range of practitioners, including those in charge of graduate pedagogy or teacher development, as well as writing teachers and WPAs considering ways to build greater equity into how writing is assessed and taught in their classrooms and programs. Reviews of the following texts are represented in this issue:

• Failing Sideways: Queer Possibilities for Writing Assessment by Stephanie West-Puckett, Nicole I. Caswell, and William P. Banks—reviewed by N. Claire Jackson (SUNY Geneseo)
• Teachers Talking Writing: Perspectives on Places, Pedagogies, and Programs by Shane A. Wood—reviewed by Virginia M. Schwarz (San Francisco State University)

We are thankful for the energy and hard work of our two reviewers for this issue and we hope their reviews bring renewed attention to these texts and help our readers discover new scholarship to enrich their work.

As always, we are interested in recruiting new reviewers; you can be added to our list by filling out this form. We’re also always interested in recommendations for new texts in writing assessment to review (self-promotion is welcome!); you can contact us at jwareadinglist@gmail.com.

Cheers!

Stacy Wittstock | Assistant Editor, Journal of Writing Assessment | Co-Editor, JWA Reading List | Marist College

Chris Blankenship | Assistant Editor, Journal of Writing Assessment | Co-Editor, JWA Reading List | Salt Lake Community College

Reviewed by N. Claire Jackson, SUNY Geneseo

West-Puckett, S., Caswell, N. I., & Banks, W. P. (2023). Failing Sideways: Queer Possibilities for Writing Assessment. University Press of Colorado.

Recent years have seen an increase in scholarship attending to anti-racist (e.g. Baker-Bell, 2020; Inoue, 2017; 2022; Inoue & Poe, 2012) and anti-ableist (e.g. Carillo, 2021; Kryger & Zimmerman, 2020) writing assessment practices. Failing Sideways by Stephanie West-Puckett, Nicole I. Caswell, and William P. Banks adds to this body of scholarship on equitable writing assessment by considering how we might queer writing assessment. Queer theory and writing assessment are, admittedly, questionable bedfellows. Karen Kopelson (2013) argued that it might not be possible to reconcile queer theory and writing program administration because of the former’s deliberate “turn away from pragmatism or utility, from the legitimate and legitimated, from institutions and social organizations and progress” (p. 207), and this raises similar questions about why we would want to wed queer theory and writing assessment. In the words of West-Puckett, Caswell, and Banks: “what could possibly be queer about assessment?” (p. 15). Yet for them this irreconcilability is the point. Rather than cataloguing so-called queer assessment practices, Failing Sideways provides a theoretical approach to assessment which resists dominant narratives around writing, teaching, and learning and instead centers the agency of writing teachers and students.

The first two chapters explicate their queer theoretical framework for writing assessment. Chapter 1 links the cultural discourses around “learning loss” that emerged during the COVID-19 pandemic to No Child Left Behind and to “Why Johnny Can’t Write” (Sheils, 1975) to remind us many of our assessment practices are driven by this persistent fear of failure. That is, as concerns with grade inflation make clear, we simply cannot imagine a system in which everyone, or even most people, succeed. Thus, Failing Sideways asks us to queer assessment by disidentifying (Muñoz, 2013) with normative assessment paradigms and embracing the disposition of an “assessment killjoy” (à la Ahmed’s, 2017, feminist killjoy). The assessment killjoy “denies convenience, unthinking happiness, and normative investment in the illusion of objectivity” (p. 17) by questioning the linear success narratives and systems of power and privilege that undergird our assessment practices.

Chapter 2 then begins with a discussion of the affects that circulate around academic success (such as pride, joy, anxiety, fear, shame, failure) and asks what assessment might look like if it were to instead center passion, excitement, and desire. This discussion of affect frames their concept of Queer Validity Inquiry (QVI). QVI extends the work of Critical Validity Inquiry (Perry, 2012) by emphasizing student and teacher voices, assuming that our assessments have consequences, and interrogating who benefits, and who is harmed, by our assessment practices. QVI resists the success/failure binary, recognizing that following the “wrong path” can reveal interesting insights into how writing and writing instruction impacts/is impacted by people differently in different contexts. Most importantly, QVI takes the affective experience of writing assessment seriously in order to prioritize the embodied experiences of writers.

Chapter 3 introduces the first of four failure-oriented principles of QVI: failing to be successful. This principle resists normative notions of success and views failure as an opening of opportunity. Our widespread fear of failure, they argue, arises from a desire to avoid shame, which limits the potential of transformative assessment practices. For example, they explore the ways portfolio assessment is often taken up uncritically because our scholarship has already marked it as a “successful” practice. Instead of assuming what was successful in one context will be successful in another, QVI encourages us to embrace this fear of shame and explore other options. They discuss several of their experiences with both programmatic and classroom assessment practices which demonstrate what this might look like, such as a programmatic assessment in which participants were asked to assume the common rubric failed to capture something important about student writing. This approach, they explain, risks potential shame for instructors, as it may reveal they are doing a “bad” job teaching writing. However, they argue it led to productive discussions about what is happening across writing classes and provided space for participants to interrogate their assumptions about writing and learning. What the various examples in this chapter share is a resistance to reducing writing to easily measurable metrics, instead exploring the affective experiences around writing as a way to embrace agency

In Chapter 4 they discuss the second principle of QVI: Failing to be commodified. They argue that writing traits and even assessment itself have become commodities, which they demonstrate through a discussion of the 6+1 Trait® Writing Rubric. While this rubric started as a well-meaning assessment practice, it has transformed into a decontextualized commodity sold as a “quick fix” to writing instruction which merely creates a closed assessment loop. This discussion is especially useful in demonstrating the ways acontextual notions of “good writing” can become entrenched within our assessment practices even when we problematize such static notions of “good writing.” QVI resists such commodification by turning towards assessment practices that highlight vulnerability and consent through collaborative negotiations of the parameters of what matters in writing in the hyperlocal situation in which it is being used, thus resisting norming and embracing the fact different readers will understand texts differently. Once again, they provide several programmatic and classroom assessment examples demonstrating ways they have enacted this principle. What these practices emphasize is not whether or not a student met a particular skill, but the “labor, relationships, emotions, and histories” (p. 142) of the writer as they engage with the various readers of their texts. Writing instructors interested in how they can assist students in moving beyond writing solely for the teacher will find this chapter especially compelling.

In Chapter 5, the focus shifts towards dissensus and radical justice via failing to be reproduced. Drawing on Edelman’s (2004) critique of reproductive futurity, they argue that our linear success models reproduce (hetero)sexist, racist, and classist writing assessment practices. They draw a connection between these linear success narratives and normative grade distribution models, which, despite extensive critiques, often still inform our day-to-day practices. QVI, however, refuses this reproductive futurity by privileging the unexpected and frequently interrogating our current expectations. To demonstrate how this might work in the classroom, they discuss a digital badging approach to labor-based grading which allows students to pursue a variety of pathways based on their own desires. Turning towards programmatic assessment, they discuss two examples of how sampling demographic data differently can bring insights into the affective experiences of minoritized students that are hidden by our normal data aggregation practices. Central to this principle of QVI, then, is continuing to ask what impacts our practices have on students/writers that are not immediately apparent within our current assessment models. This chapter is an especially important read for WPAs, as it reveals how complacency with our current programmatic assessment models can disadvantage many of our students.

Chapter 6 explains the fourth and final principle: Failing to be mechanized. In this chapter they argue that large-scale programmatic assessment practices focused on objective criteria and normed writers convey writing success as an individualized rather than systemic matter, ignoring the material conditions of the students producing those texts. In failing to be mechanized, QVI embraces the relationality of writing, considering all the human and nonhuman actants that contributed to the writing process. They present game play as a useful way to subvert this mechanization, providing examples such as origami fortune tellers which map the various people and materials that influenced a particular writing project, or learner stories which narrativize a writer’s experiences in a class and resist ranking students against each other. Importantly, these stories can be shared with those being assessed, thus requiring us to be accountable to the students impacted by our assessments.

To conclude, Chapter 7 extends an invitation to readers to join their Queer Assessment Collaborative Killjoy Army. They recognize the chapters in this book do not provide an easy roadmap for how to queer our assessment practices. Yet this is the point: QVI is meant to be messy and time intensive, just like writing itself. They argue this work will not only lead to more equitable and fair assessments, but it will also provide tactics for pushing back on normative institutional assessment paradigms, which they demonstrate by discussing how they have resisted artificially imposed external assessment demands. Thus, readers who question the practicality of the assessment paradigms this book offers should pay especially close attention to this concluding chapter. Lastly, as they ask us to take up the ethos of the assessment killjoy, they remind us that the work of designing fair and equitable assessments is never complete, as assessments need to attend to the specific students we are working with at that specific moment.

In short, through their extensive discussions of how even our most well-meaning assessment tools and practices become normativized, Failing Sideways makes a crucial contribution to our field’s discussions of socially just writing assessment. As they repeatedly note, the book fails to offer any “quick fix” to assessment, or any practices we can immediately implement. However, it does provide a useful framework for developing a new disposition towards assessment that centers the ever-changing needs of our students. Because they focus on both programmatic and classroom assessment, WPAs and writing instructors alike will find this book useful for considering how we might rethink our approaches to assessment.

References

Ahmed, S. (2017). Living a feminist life. Duke University Press.

Baker-Bell, A. (2020). Linguistic justice: Black language, literacy, identity, and pedagogy. Routledge.

Carillo, E. (2021). The hidden inequities in labor-based contract grading. University Press of Colorado.

Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. The WAC Clearinghouse; Parlor Press.

Inoue, A. B. (2022). Labor-Based Grading Contracts: Building Equity and Inclusion in the Compassionate Writing Classroom, 2nd ed. The WAC Clearinghouse; University Press of Colorado.

Inoue, A. B., & Poe, M. (Eds.). (2012). Race and writing assessment. Peter Lang.

Kopelson, K. (2013). Queering the writing program: Why now? How? And other contentious questions. Writing Program Administration, 37(1), 199-214.

Kryger, K., & Zimmerman, G. (2020). Neurodivergence and intersectionality in Labor-Based Grading contracts. Journal of Writing Assessment, 13(2).

Muñoz, J. E. (2013). Disidentifications: Queers of color and the performance of politics (Vol. 2). University of Minnesota Press.

Perry, J. W. (2012). Critical validity inquiry. In K. M. Powell & P. Takatoshi (Eds.), Practicing research in writing studies: reflective and ethically responsible research (pp. 187-211). Hampton Press.

Reviewed by Virginia M. Schwarz (she/her), San Francisco State University

Wood, S. A. (2023). Teachers Talking Writing: Perspectives on Places, Pedagogies, and Programs. WAC Clearinghouse.

As a first-time graduate student in 2006-2008, my favorite “text” was Todd Taylor’s Take 20 (2008), a now out-of-print DVD that explored 20 questions that were, at that time, central to the teaching of writing. The disc featured 22 writing studies scholars talking about their classroom teaching, including Brian Huot, Paul Kei Matsuda, Jacqueline Jones Royster, Mike Rose, and Nancy Sommers. They give us insight into what they wished they would have known as new teachers, and they share strategies for those entering the field and the classroom for the first time. Take 20 gave me a felt sense of community, like I was joining a conversation and these teachers would help me learn. Teachers Talking Writing: Perspectives on Places, Pedagogies, and Programs reminds me of that DVD.

Teachers Talking Writing: Perspectives on Places, Pedagogies, and Programs (TTW) is Shane A. Wood’s full-length monograph inspired by his interviews with writing teachers on his podcast, Pedagogue. As of April 2024, 164 podcast episodes have been published on the Pedagogue website along with their full transcripts. In TTW, Wood groups and synthesizes many of those early interviews to create a rich dialogue around a specific context or topic in the field. These 52 perspectives are strategically organized into three sections: Places, Pedagogies, and Programs.

As a long time listener of Pedagogue, my primary question picking up TTW was whether it would offer something new that Pedagogue does not. Would TTW simply be a reprinted series of interview transcripts? I was really pleased that Wood addresses these questions in his introduction when he describes his own thought process for writing the book and imagines its potential uses. For context, I’m approaching this review of TTW as a former community college instructor and current assistant professor who now works in composition MA and certificate programs. This means that my own interest in Wood’s work stems from approaches to and issues within faculty development and graduate student education. In addition to those frames, readers of JWA may also be interested in TTW because of Wood’s attention to assessment specifically and larger systems of valuing (and devaluing) more broadly.

For Wood, TTW is both an extension and a disruption of a genre, the composition anthology. In traditional anthologies, Wood argues, well-known R1 scholars typically drive conversations about teaching writing (p. 2). Additionally, most of the chapters are solo-authored and therefore can only capture a single perspective (p. 2). While Wood acknowledges how important these texts have been for the field and for his own development, TTW, in contrast, centers teachers, graduate students, junior faculty, faculty across rankings, and faculty from various kinds of institutions (p. 10). Wood also reminds readers that historical silences in the field exist due to racism (p. 10). He hopes that “TTW (and Pedagogue) can challenge gaps in scholarship and further examine power and race” (p. 2, 10). Not only does Wood want to flip the script on who is included, but he was also mindful of opening these conversations to a larger audience by making this text open access. 

Wood images that TTW can be used in a variety of ways, including alongside the Pedagogue podcast. In fact, he describes his book and his podcast as “interconnected” (p. 2). For example, at the top of each printed conversation in TTW, there is a reference and timestamp corresponding to the full audio interview on Pedagogue. This invites readers and listeners to go back and forth between the two mediums or engage them at the same time. Wood also states that readers might use TTW alongside other texts. For example, we might read one of the featured teacher’s own books and then reference a specific interview in TTW to better understand them or their work (p. 14). What makes TTW different from just a chronological archive of interview transcripts is Wood’s strategic grouping of these conversations and the framing he provides in both the beginning and end of each chapter. “I see Pedagogue as a monologue and TTW as the full script for a play,” Wood writes (p. 4). “Pedagogue focuses on individual actors; each episode is a center stage spotlight on teacher-scholars talking about their teaching and institutional context. TTW, on the other hand, is interwoven scenes that comprise a full production and collaborative performance that consists of a much larger plot.” (p. 4). Readers will find that Wood concludes each chapter with a “denouement” where he highlights some of the conversational threads present across multiple interviews. Each chapter ends with a series of questions for further thinking.

TTW is divided into three sections, Places, Pedagogies, and Programs. The first section, Places, has four chapters: (1) Pathways and Reflections on Teaching, (2) Two-Year Colleges, (3) Historically Black Colleges and Universities, and (4) Hispanic-Serving Institutions. The second section of TTW focuses on Pedagogies and includes five chapters: (1) Classroom Writing Assessment, (2) Multimodality, (3) Social Justice, (4) Disability Studies, and (5) Community Literacies. Finally, Wood’s third section, Programs, entails five chapters: (1) Writing Program Administration, (2) Basic Writing, (3) Second-Language Writing, (4) Writing Across the Curriculum, and (5) Writing Centers. Wood provides a summary of each section in the TTW introduction and, for me, the chapters and their pieces (context, excerpts, denouement, thinking questions) are easy to locate, navigate, skim, and read in any order.

Assessment, as a focus topic, has its own chapter under Pedagogies. Wood constructs this conversation from his Pedagogue interviews with Nancy Sommers, Chris M. Anson, Jennifer Grouling, and Asao B. Inoue. The questions he selects and the excerpts he includes primarily center classroom writing assessment and, more specifically, feedback and response. Sommers shares some of her commenting strategies and how they emerge from the shared language and context of each class; for example, she thinks about the writer in addition to the writing, she talks to students about commenting before they receive comments on their first assignment, and strives to give feedback that is reflective both specific conversations with students and her own values as an educator (p. 113-116). Next, Wood includes Anson who shares that he became interested in students’ perceptions of teacher response as a graduate student when his own teacher used cassette tapes to record feedback (p. 117). This leads into Grouling’s interview and research on how different learning management systems (Canvas, Blackboard) and material technologies (iPads) can impact the kinds of comments teachers make on student writing (p. 119-121). Asao B. Inoue is the final person Wood includes, and in these excerpts, Inoue talks about centering labor in an assessment ecology to “shift [what is problematic] away from the politics of language and the politics of identity… to the politics of economics and how much time do I have” (p. 124). Inoue also discusses fairness and assessment, his grading contract negotiation process, and the importance of students’ participation in shaping classroom practices (p.122-125).

I can imagine using this chapter in many of the ways Wood describes in his introduction. For example, I would be excited to pair the first three interviews (Sommers, Anson, & Grouling) with chapters from Bad Ideas About Writing (Ball & Loewe, 2017) for graduate students, faculty, and as a reflective experience for myself. Taken together, these texts would likely start a productive discussion about a variety of feedback approaches and practices. Also, if I were using TTW to explore someone’s body of work, Inoue’s interview on Pedagogue and the excerpts here in TTW could work alongside and bring another layer to theoretical texts like Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future (Inoue, 2015) and Labor-Based Grading Contracts: Building Equity and Inclusion in the Compassionate Writing Classroom (Inoue, 2019). If someone is beginning to learn about teaching writing and writing assessment, TTW offers more context and support than individual Pedagogue episodes alone. The thinking questions that Wood includes at the end of this chapter (or versions of them) could probably work in most classrooms or workshops, and one of the questions directly asks readers to reflect on how assessment and teacher response are connected to linguistic justice and diversity (p. 126).

In his own words, TTW “centers conversation as a tool for building knowledge and community, and prioritizes dialogue, inclusivity, and accessibility” (p. 14). I also value Wood’s critique of anthologies in the field and his appreciation for the texts that made us but also his recognition that these are inherently narrowed, often privileging R1 tenure-track faculty. For example, in Naming What We Know: Threshold Concepts of Writing Studies (Adler-Kassner, L. & Wardle, 2015), one well-known scholar is tasked with defining each key disciplinary term. Coming from a two-year college, I often found myself outside the “we” in many canonical writing studies handbooks. While there are going to be gaps in any collection, I agree with Wood that dialogue across perspectives can serve as an intervention in dominant disciplinary narratives and add important nuance to conversations about teaching.

Ultimately this book gave me ideas for additional ways to use the podcast. I also think this could be a valuable tool for new teachers of graduate students, like myself, who might be conceptualizing course design or perhaps for senior faculty who want to reinvent their “introduction to composition and rhetoric” course. Wood organizes these interviews in a conversational way and emphasizes inquiry. When students ask me for teaching or career advice, I often tell them that I will share what I think but only if they promise to talk to at least three other people. TTW encourages and embodies this spirit of discovery, acknowledging that all perspectives are limited while also celebrating each as an important part of a larger story. Wood’s book intervenes in a genre that lends itself to codifying knowledge, best practices, and celebrating particular institutions and stories. I appreciate Wood’s effort to include a range of voices in TTW, and I can imagine that some teachers and students will find this accompanying text very valuable for working through important issues in composition. For me, Wood’s multimodal, multivocal work feels like a necessary update to Take 20.

References

Adler-Kassner, L. & Wardle, E. (2015). Naming What We Know: Threshold Concepts of Writing Studies. Utah State University Press.

Ball, C. & Loewe, D. M. (2017) Bad Ideas About Writing.

Inoue, A. B. (2015) Antiracist Writing Assessment Ecologies: Teaching and Assessing Writing for a Socially Just Future. WAC Clearinghouse.

Inoue, A. B. (2019). Labor-based grading contracts: Building equity and inclusion in the compassionate writing classroom. WAC Clearinghouse.

Taylor, T. (2008). Take 20. Bedford/ St. Martin’s.

Greetings!

The Journal of Writing Assessment’s Reading List is excited to release our Summer 2023 Issue!

Our reviews in this issue explore four recent books related to assessment across a spectrum of educational contexts, including K-12 classrooms, two-year colleges, and four-year institutions. The texts also cover a range of assessment areas, including writing placement, writing in and across the disciplines, equitable classroom assessment, and high-stakes standardized testing in K-12 contexts. Reviews of the following texts are represented in this issue:

• Writing Placement in Two-Year Colleges: The Pursuit of Equity in Postsecondary Education, edited by Jessica Nastal, Mya Poe, and Christie Toth—reviewed by Megan Friess (Cypress College)
• Reframing Assessment to Center Equity: Theories, Models, and Practices, edited by Gavin W. Henning, Gianina R. Baker, Natasha A. Jankowski, Anne E. Lundquist and Erick Montenegro—reviewed by Stephanie Hedge (University of Illinois, Springfield)
• Assessing Writing to Support Learning: Turning Accountability Inside Out by Sandra Murphy, Peggy O’Neill—reviewed by Jeremy Levine (University of Massachusetts, Amherst)
• Improving Outcomes: Disciplinary Writing, Local Assessment, and the Aims of Fairness edited by Diane Kelly-Riley, Norbert Elliot—reviewed by Anthony Lince (University of California, San Diego)

We are thankful for the energy and hard work of all of our reviewers and we hope their reviews bring renewed attention to these texts and help our readers discover new scholarship to enrich their work. We’d also like to thank our amazing team of graduate assistant editors: Kathleen Kryger (University of Arizona), Jennifer Burke Reifman (University of California, Davis), Tiffany Smith (Georgia State University), and Sarah Stetson (Brown University).

As always, we are interested in recruiting new reviewers; you can be added to our list by filling out this form. We’re also always interested in recommendations for new texts in writing assessment to review (self-promotion is welcome!); you can contact us at jwareadinglist@gmail.com.

Cheers!

Stacy Wittstock | Assistant Editor, JWA Reading List | University of Nevada, Las Vegas

Chris Blankenship | Assistant Editor, JWA Reading List | Salt Lake Community College

Reviewed by Anthony Lince, University of California, San Diego

Kelly-Riley, D., & Elliot, N. (Eds.). (2020). Improving outcomes: Disciplinary writing, local assessment, and the aim of fairness. Modern Language Association.

When it comes to assessment, our field is currently having challenging, but much-needed, conversations—some of which are focused on equity, linguistic justice, and student agency. Asao Inoue (2019), for example, has pushed back against traditional grading practices and, is instead, in favor of labor-based grading contracts, which, Inoue asserts, “attempt to form an inclusive, more diverse ecological place, one that can be antiracist and anti-White supremacist by its nature (p. 13). These conversations around assessment, however, aren’t exclusive to our field. In Susan D. Blum’s (2020) edited collection, Ungrading: Why Rating Students Undermines Learning (and What to Do Instead), educators in the humanities, social sciences, and STEM fields all wrestle with how they can move away from grading practices that are punitive and not student-centered. (For an excellent overview of this book, check out Michelle Tram Nguyen’s recent review on the JWA Reading List.)

Improving Outcomes: Disciplinary Writing, Local Assessment, and the Aim of Fairness—a collection edited by Diane Kelly-Riley and Norbert Elliot—contributes to this important conversation on assessment with a focus on fairness and assessment across the disciplines. Kelly-Riley and Elliot note that, “within this collection, fairness operates as an integrative principle” (p. 1). Though, fairness isn’t thought of as a monolithic idea that can be applied to all fields. Instead, as Anne Ruggles Gere makes clear in her foreword, “by recognizing and valuing the discourses of a given discipline, writing assessment can enact fairness in assessment rather than applying inflexible standards to all fields” (p. vi). She continues: “the best assessment is constructed locally, and, for college students, the disciplines in which they enroll become a local context” (p. vi). Naturally, then, to discuss this varied, and situated, idea of fairness, the contributors in this collection span the disciplines—from nursing to engineering, writing studies, and architecture—and are from a range of academic contexts: two- and four-year to public and private institutions. Constructed around putting “fairness at the center” of writing instruction and assessment (p. 5), this collection is divided into four parts: “Values,” “Foundational Issues,” “Disciplinary Writing,” and “Location.”

The contributors of part one, “Values,” all examine the unique needs of students within specific academic contexts and how educational values should be tied to those needs. Mya Poe begins part one with her essay, “A Matter of Aim: Disciplinary Writing, Writing Assessment, and Fairness.” She turns to assessment research to “examine two common frames for writing assessment in the disciplines—program accreditation and classroom research” (p. 17), concluding that considerations around student fairness are often ignored in both frames. Ruth Osorio’s essay, “A Disability-as-Insight Approach to Multimodal Assessment,” lays out ways in which a disability-as-insight model can be used “as a path that merges fairness—designing assessments that allow for diverse and flexible methods for achieving the primary goal of an assignment—and social justice” (p. 29). Brooke A. Carlson and Cari Ryan, in “Fairness as Pedagogy: Uniformity, Transparency, and Equity through Trajectory-Based Responses to Writing in Hawai’i,” use rubrics as a tool to promote fairness by being transparent with students about the evaluative methods in which they will be graded.

The contributors of part two, “Foundational Issues,” outline educational measurement as socially situated. The first essay argues for seeing assessment as an evidentiary argument—with a focus on students developing competencies in valued activities (Mislevy). Benander and Refaei, in their essay, detail how their basic writing courses have outcomes that are fairly assessed “through shared rubrics tailored to the interests of each student” (p. 67). The next essay explores how peer-feedback can be embedded in classrooms as a means to promote fairness (Hart-Davidson and Meeks). Erick Montenegro, in the penultimate essay of part two, asserts that “assessment efforts must become culturally responsive” to better understand the learning gains made by students (p. 93). The last essay in part two argues for faculty members to learn about various disciplinary perspectives to create shared learning outcomes at specific institutions (Schneider and Hennings).

In part three, “Disciplinary Writing,” the scholars focus on assessments that are situated within their specific educational contexts. The first essay argues for a strengthened connection between high school and college literacies (Farris). The next essay’s authors discuss how they use evidence-based assessment in their first-year composition program to promote programmatic fairness (Buyserie, Macklin, Frye, and Ericsson). Singer-Freeman and Bastone, in their essay, argue for reflective writing in a child development course to help students think deeply about their own lives and the course content. In an architecture writing course, Hogrefe and Briller argue for reflective practices that can help their diverse cohort of students. In their essay, Maneval and Ward discuss how the incorporation of nursing-specific writing genre assignments in nursing classes could elevate writing itself as a practice. Williams, in the last essay of part three, discusses issues of fairness as it relates to assessment within science, engineering, and mathematics courses.

“Location,” part four, closes the collection by having essays that move beyond traditional four-year institutions. Rasmussen and Reid consider questions around transfer and equal opportunity at their two-year college. Whithaus, in the next essay, considers how “localized assessments can attend to fairness, as well as validity and reliability,” not only face-to-face but online and in hybrid classes as well (p. 213). Rhodes, in the final essay of part four and in this edited collection, discusses accreditation as something that can, and should, “affirm institutional commitment to fairness for students’ access to, and achievement of, quality learning” (p. 225).

Taken together, there were parts of this collection that strongly resonated with me. A disability-as-insight approach for multimodal assessment (Osorio) helps me consider the ways I can construct my classrooms and assignments to best help all learners succeed, especially students who learn in non-normative ways. Mya Poe’s essay was also illuminating as she illustrated the racial harm that placement tests can have on certain students. And Hogrefe and Briller, in their essay on an architecture writing program, provided a wonderful message for any teacher or program director to take away: to have authenticity of curriculum, “students [should be] placed at the center of our efforts and treated as colleagues” (p. 170).

On the other hand, the essays in this collection that had a focus on using rubrics weren’t, for me, all that convincing. Those authors claimed that rubrics can be fair because they are transparent for students. However, I question this claim, and I wonder how transparent racially situated biases can be through the use of rubrics. Furthermore, in my experience, rubrics seem to erase individuality, not promote diverse thinking. If fairness is the goal, rubrics seem to hinder that outcome.

With that noted, the conversations in this book centered on fairness and assessment are crucial for our field and others to have. Any rhetoric and writing studies scholar can find engaging ideas here, but I’d specifically recommend this collection to new rhet/comp scholars entering the field and/or to those in other disciplines wanting to integrate writing into their programs—and, by extension, assessment of that writing—with the aim of being fair.

References

Blum, S. (2020). Ungrading: Why rating students undermines learning (and what to do instead). West Virginia University Press.

Inoue, A. B. (2019). Labor-based grading contracts: Building equity and inclusion in the compassionate writing classroom. WAC Clearinghouse https://doi.org/10.37514/PER-B.2019.0216.0

Nguyen, M. (2022). [Review of the book Ungrading: Why rating students undermines learning (and what to do instead)]. The Journal of Writing Assessment Reading List.

Reviewed by Jeremy Levine, University of Massachusetts, Amherst

Murphy, S., & O’Neill, P. (2023). Assessing writing to support learning: Turning accountability inside out. Routledge.

Sandra Murphy and Peggy O’Neill’s (2023) Assessing Writing to Support Learning: Turning Accountability Inside Out synthesizes existing research on writing assessment, psychometrics, and writing pedagogy to argue that teachers should be at the center of the school accountability system. Foregrounding formative assessment processes such as portfolio grading, Murphy and O’Neill propose a framework through which ecological writing assessment (which has been applied at the post-secondary level, per Wardle and Roozen 2012; Inoue 2015) can be brought to K-12 instruction. In the book’s first chapter, they argue that such a pivot will reduce the extent to which high-stakes assessment narrows writing curricula, account for a fuller picture of writers’ knowledge aligned with modern research, and include teachers as active decision-makers in writing assessment. This claim illuminates the administrative and policy risks of hitching the K-12 writing assessment wagon to standardized tests: because of their limited view, tests can misguide administrators and the public about what our students know about writing. On the teaching side, the emphasis on the narrowed curriculum could also include examination of the contextual nature of testing’s influence on instruction (McCarthey, 2008) and how teachers mediate testing expectations through their own goals for writing (e.g. Wahleithner, 2018). These local concerns shift the book’s exigence slightly: teachers are already making writing assessment their own; a more productive policy paradigm would build on this teacher agency, rather than create obstacles for it.

Chapter Two is a crash course in writing assessment, overviewing the fundamental concepts of reliability and validity. Validity is of particular interest to Murphy and O’Neill, who make two validity-based critiques of high-stakes testing. The first is that the accountability system must take consequential validity seriously: that the purpose of administering a test affects how teachers and students approach it, meaning the curricular changes that accompany high-stakes testing are a threat to the test’s validity. Second, standardized testing has weaknesses in terms of construct validity: the extent to which a test measures what it claims to. The construct validity critique is built on the concept that student text is not necessarily a stand-in for student writing knowledge, as a student’s ability to produce a specific genre under testing circumstances cannot speak to their rhetorical flexibility or approach to writing across genres, purposes, or settings. This claim about construct validity helpfully builds on the growing body of research that locates substantial portions of writing development as taking place off the page, including concepts such as dispositions and identities (see Driscoll & Zhang, 2022). The importance of each of these concerns is made clear in Chapter Three, which focuses on evolving theories of writing and writing instruction. Accounting for both social and cognitive theories of writing, O’Neill and Murphy offer an overview of writing concepts (e.g. writing as expression, writing as a product, writing as a social activity, etc.) and instructional practices (writing for a real audience, building genre knowledge, participating in peer review, reflecting). Composition researchers will surely recognize these lists of concepts, but they do important work in demonstrating how out-of-step a high-stakes exam is with theories of writing instruction (a blow to its consequential validity) and to how writing is understood (a blow to its construct validity).

With these flaws in high-stakes assessment established, the rest of the book pivots toward solutions. The first of two goals in Chapter Four is to outline classroom-scale models of formative assessment that give students opportunities to reflect on their own writing processes. To illustrate the rigor of such formative assessments, and demonstrate their promise of improving metacognition, O’Neill and Murphy offer several examples of self-assessment rubrics that may help teachers and students identify key facets of the writing process to focus on. The second goal of Chapter 4 is to describe approaches for large-scale assessment practices that align with the cognitive and social characteristics of writing described in the third chapter. This means conceiving of writing as a task- or context-specific activity, which in testing circumstances might involve portfolio assessment, assessments that integrate reading and writing, collaborative writing, and digital or multimodal writing tasks. These recommendations culminate in the authors’ invocation for accountability to be turned “inside-out,” putting the complexity of writing, and the needs of students, at the forefront of writing assessment on a large scale, rather than prioritizing the psychometric approach of standardization and controlling variability. These arguments for rewriting large-scale writing assessment lead to questions about what happens when these measures are attached to accountability systems. For example, portfolio assessment at the state level may absorb all student writing into a bureaucratic system (Scott, 2008), or schools may feel undue pressure to improve, say, “writing motivation” scores, and as a result, could focus more on scores than on actual writing motivation (see Koretz, 2017). O’Neill and Murphy make a strong case for why these elements must be included in large-scale assessment; the question is what the implications might be once that happens.

Chapters Five and Six offer strategies for bringing the recommendations from Chapter Four into reality. Chapter Five focuses on methods for redesigning writing assessment, arguing that teachers need to be at the center of assessment processes because teachers are ultimately responsible for implementing classroom changes. To make this change a reality, O’Neill and Murphy propose investing in professional development, involving teachers in assessment design, and supporting collaboration across levels of education. The sixth and final chapter of the volume focuses on an ecological model of writing assessment. Building on the work of Inoue (2015) and Wardle and Roozen (2012), both of whom focus on assessment ecologies at the post-secondary level, the authors offer an invocation for similar frameworks to make their way into K-12 schools. Combined with the emerging psychometric concept of ecological validity, this chapter’s focus on ecologies creates a “springboard for action” (p. 192) that can mobilize teachers and researchers toward shifting the terms of control for assessment and accountability in the United States. This strategy combines rather nicely with parallel calls for reform in education studies, which suggest student surveys (Schneider et. al, 2021) or inspectorates (Berner, 2017) may be more productive (and ecologically-minded) assessment systems. As O’Neill and Murphy conceptualize it, writing can be a productive window into school life, thereby giving it the potential to be especially useful in these imagined reforms.

In total, Assessing Writing to Support Learning: Turning Accountability Inside Out offers conversation-starting concepts for multiple audiences. For policymakers at school, local, and even state levels, it illustrates how modern conceptions of accountability are out-of-step with best practices in writing instruction and assessment. For instructors at the primary, secondary, and post-secondary levels, it invites reflection around how classroom assessment practices can be used to foster students’ learning about writing, and when or how such assessment practices can become disconnected from assessment. For researchers on writing, it offers a framework for conceptualizing validity on ecological terms and invites future inquiry on the intersection of classroom assessment and policy concerns. Primarily grounded in research and concepts from writing studies, its connections to education reform – implementation, accountability, and possibilities for reform – leave lingering questions regarding how the proposed ecological model of assessment can be implemented as policy. At its core, the text is a reminder that classrooms are about relationships between students and teachers, and this relationship — not the concerns of parties outside of that room — should be at the center of conversations about learning.

References

Berner, A (2017). Would School Inspections Work in the United States? Johns Hopkins School of Education, Institute for Education Policy.

Driscoll, D. L., & Zhang, J. (2022, March). Mapping long-term writing experiences: Operationalizing the writing development model for the study of persons, processes, contexts, and time. In Composition Forum (Vol. 48). Association of Teachers of Advanced Composition.

Inoue, A. B. (2015). Antiracist writing assessment ecologies: Teaching and assessing writing for a socially just future. Parlor Press.

Koretz, D. (2017). The testing charade: Pretending to make schools better. University of Chicago Press

McCarthey, S. J. (2008). The impact of No Child Left Behind on teachers’ writing instruction. Written Communication, 25(4), 462-505

Murphy, S., & O’Neill, P. (2022). Assessing writing to support learning: Turning accountability inside out. Taylor & Francis.

Schneider, J., Noonan, J., White, R. S., Gagnon, D., & Carey, A. (2021). Adding “student voice” to the mix: Perception surveys and state accountability systems. AERA Open, 7, 1-18.

Scott, T. (2008). “Happy to comply”: Writing assessment, fast-capitalism, and the cultural logic of control. The Review of Education, Pedagogy, and Cultural Studies, 30(2), 140-161.

Wahleithner, J. M. (2018). Five portraits of teachers’ experiences teaching writing: Negotiating knowledge, student need, and policy. Teachers College Record, 120(1), 1-60

Wardle, E., & Roozen, K. (2012). Addressing the complexity of writing development: Toward an ecological model of assessment. Assessing Writing, 17(2), 106-119.