Category: Uncategorized

Part I: Review of Handbook of Automated Essay Evaluation: Current Applications and New Directions. Eds. Mark D. Shermis and Jill Burstein

Shermis, M., & Burstein J. (2013). Review of Handbook ofAutomated Essay Evaluation: Current Applications and New Directions. New York, NY: Routledge.

By Lori Beth De Hertogh, Washington State University

The Handbook of Automated Essay Evaluation: Current Applications and New Directions edited by Mark D. Shermis, University of Akron, and Jill Burstein, Educational Testing Service, features twenty chapters that each deals with a different aspect of automated essay evaluation (AEE). The overall purpose of the collection is to help professionals (i.e. educators, program administrators, researchers, testing specialists) working in a range of assessment contexts in K-12 and higher education better understand the capabilities of AEE. It also strives to demystify machine scoring and to highlight advances in several scoring platforms.

The collection is loosely organized into three parts. Authors of the first three chapters discuss automated essay evaluation in classroom contexts. The next section examines the workflow of various scoring engines. In the final section, authors highlight advances in automated essay evaluation. My two-part review generally follows this organizational scheme, except that I begin by examining the workflow of several scoring systems as well as platform options. I then review how several chapters describe potential uses of AEE in classroom contexts and recent developments in machine scoring.

The Handbook of Automated Essay Evaluation devotes considerable energy to explaining how scoring engines work. Matthew Schultz, director of psychometric services for Vantage Learning, describes in Chapter Six how the IntelliMetric™ engine analyzes and scores a text:

The IntelliMetric system must be ‘trained’ with a set of previously scored responses drawn from expert raters or scorers. These papers are used as a basis for the system to ‘learn’ the rubric and infer the pooled judgments of the human scorers. The IntelliMetric system internalizes the characteristics or features of the responses associated with each score point and applies this intelligence to score essays with unknown scores. (p. 89)

While the methods platforms like IntelliMetric use to determine a score are slightly different, they all employ a multistage process, which consists of four basic steps:

•  receiving the text,
• using natural language processing to parse text components such as structure, content, and style,
• analyzing the text against a database of previously human- and machine-scored texts,
• producing a score based on how the text is similar or dissimilar to previously rated texts.

In Chapter Eight, Elijah Mayfield and Carolyn Penstein Rosé, language and technology specialists at Carnegie Mellon University, demonstrate how this four-step process works by describing the workflow of LightSIDE, an open source machine scoring engine and learning tool. In doing so, they illustrate how the program is able to match or exceed “human performance nearly universally” due to its ability to track and develop large-scale aggregate data based on text data. Mayfield and Rosé argue that this feature allows LightSIDE to tackle “the technical challenges of data collection” in diverse assessment contexts (p. 130). They also emphasize that this capability can help users curate large-scale data based on error-analysis. Writing specialists can then use this information to identify areas (i.e. grammar, sentence structure, organization) where students need instructional and institutional support.

Chapter Four, “The e-rater® Automated Essay Scoring System,” provides a “description of e-rater’s features and their relevance to the writing construct” (p. 55). Authors Jill Burstein, Joel Tetreault, and Nitin Madnani, research scientists at Educational Testing Service, stress that the workflow capabilities of scoring systems like e-rater or Criterion (a platform developed by ETS) make them useful tools for providing students with immediate, relevant feedback on the grammatical and structural aspects of their writing in addition to being useful in administrative settings where access to aggregate data is critical (pp. 64-65). The authors argue that e-rater’s ability to generate a range of data make it an asset in responding to both local and national assessment requirements (p. 65).

In Chapter Nineteen, “Contrasting State-of-the-Art Automated Scoring of Essays,” authors Mark D. Shermis and Ben Hamner (Kaggle) offer readers a comparison of nine scoring engines’ responses to a variety of prompts in an effort to assess and compare the workflow and performance levels of each system, some of which include Intelligent Essay Assessor, LightSIDE, e-rater, and Project Essay Grade. This chapter may be particularly useful to individuals tasked with determining which type of automated evaluation system to adopt or replace. In addition, this chapter provides a brief guide to understanding how a variety of systems operate and an overview of “vendor variability in performance” (p. 337).
The Handbook of Automated Essay Evaluation: Current Applications and New Directions provides assessment scholars, practitioners, and writing teachers relevant information about the workflow of various scoring engines and how these systems’ functioning capabilities can be applied to a range of educational settings. By understanding how these systems work and their potential applications, individuals tasked with writing assessment can make more informed choices about the potential benefits and consequences of adopting automated essay evaluation.

 

JWA will be at the Research Network Forum and CCCC in Indianapolis, March 19-22, 2014.

JWA will be at the Editors’ Roundtable discussion on Wednesday, March 19, 2014 from 1:15-2:30 pm.

If you would like to talk to someone from JWA about a potential project, you can reach Peggy O’Neill at poneill1 [at] loyola [dot] edu or you can contact Jessica Nastal-Dema at jlnastal [at] uwm [dot] edu.

See you there!

Review of Building Writing Center Assessments that Matter by Ellen Schendel and William Macauley (2012). Utah State University Press.

ISBN 978-0-87421-816-9, paper $28.95; ISBN 978-0-87421-834-3 e-book $22.95

By Marc Scott, Shawnee State University

Ellen Schendel and William Macauley’s 2012 book, Building Writing Center Assessments that Matter (Building), is a co-authored text featuring an introduction and coda by both authors, three chapters authored by Macauley, three by Schendel, a brief interchapter by Neal Lerner, and an afterward by Brian Huot and Nicole Caswell. Much of Building explores how important writing assessment scholarship can apply to writing center program assessment, and often uses specific examples from the authors’ experiences directing writing centers. Schendel and Macauley’s goal in writing Building is to provide Writing Center Directors (WCDs) new to program assessment with a text that speaks specifically to the unique needs and opportunities of writing center work. While the text is geared toward assisting WCDs navigate program assessment, Building also provides assessment scholars and practitioners with important ideas and concepts for program assessment, including how to frame assessment and how to think through methodological options.

Those wishing to develop a culture of assessment at their institution can learn much from Schendel and Macauley’s text. Throughout Building, the authors use tutoring and writing processes as metaphors for assessment work. Just as writers gain invaluable insights by sharing their work with other writers, sharing assessment projects and data with peers only benefits writing assessment. Furthermore, in Writing Center scholarship and practice, tutors strive to help a writer establish a healthy writing process rather than just proofread or edit a text. When applied to writing assessment, a similar emphasis on process over product might help instructors and students engage in assessment as a reciprocal and recursive form of inquiry that improves the writer holistically, rather than a linear process with one correct approach for each context (p. xix). In addition, the assessment process—much like the writing process—benefits from careful attention to exigency, context, purpose, and audience. Using the recursion of writing processes and the context-sensitive nature of tutoring as metaphors for assessment may provide an accessible concept for colleagues reluctant to embrace assessment.

Writing assessment practitioners can also benefit from Building’s discussion of assessment methodologies. Schendel describes how Writing Center Directors should work to connect a program assessment’s methodology with each specific project’s purpose, audience, and available data. In fact, Schendel provides a useful chart that describes different forms of data a WCD might collect and explains how the data might be collected and who might collaborate in such efforts (pp. 127-131). The design of a writing assessment—be it a placement exam, a portfolio program, or a classroom assessment technique—should take the assessment’s context and purpose into account at each stage of the process, not just in analyzing results. Rather, a writing assessment should be sensitive to the context of the student and classroom. Neal Lerner’s brief interchapter helps WCDs understand how qualitative and quantitative assessment methodologies might impact assessment projects in writing centers, and his thoughts can also help persuade those reluctant to assess. He argues against “maintaining the status quo” and operating on only a “felt sense” of the work done in Writing Centers (p. 113). Classroom teachers and WPAs might also feel like they “know” their classrooms, but unless they can provide evidence through assessment for what they know, their claims will fail to persuade important stakeholders.

 

     Building, while effectively tailored to the needs of WCDs, provides assessment scholars and practitioners with useful metaphors for discussing assessment and a thoughtful discussion of assessment methodologies. The bulk of the text provides important information for those interested in programmatic assessment, but it does so by thoughtfully weaving together assessment scholarship in a way relevant to writing centers.

The Journal of Writing Assessment will be at the upcoming Writing Research Across Borders conference in Paris, France, February 19-22, 2014.

If you will be there and would like to talk with Diane Kelly-Riley, co-editor of JWA, please email her at dianek [at] uidaho [dot] edu.  We welcome WRAB presenters to adapt their writing assessment focused presentations for publication consideration to the Journal of Writing Assessment.  Presenters can find JWA submission information here.

Review of Teaching the New Writing: Technology, Change, and Assessment in the 21st- Century Classroom, by Anne Herrington, Kevin Hodgson, and Charles Moran, Editors.

Herrington, A., Hodgson, K., & Moran, C. (Eds.) (2009) Teaching the New Writing:  Technology, Change and Assessment in the 21st Century Classroom.  New York:  Teachers College Press.

By Susan Garza, Texas A&M University-Corpus Christi

Although this text was published in 2009, in 2013 as I write this review for the Journal of Writing Assessment, the use of technology continues to grow in the field of writing, and assessment seems on its way to becoming the overriding focus at all levels of education. The use of machine-scoring for writing assessment and the debate surrounding this issue is just one example of the current issues in assessment. Herrington and Moran begin the book by pointing out that while technology has propelled us forward with teaching and learning possibilities, we are being driven backwards as the drive for machine scoring unfortunately seems to be working up steam. In the last chapter, all three editors reflect on the other chapters in the book stating that their goal for the collection was to see “teachers working creatively with an expanded sense of what writing is becoming as it has accommodated emerging technologies,” and to develop an “expand[ed] sense of what the word writing might include in this new century” (198). Each chapter does have a section on assessment, although most are short and several read as if they were tacked on without much development. However, many useful rubrics/checklists that the teachers have developed in their attempts to align their activities with the curriculum standards within which they operate are included. This book could be used for a brief introduction for teachers beginning to learn about assessing writing, but it provides mostly assignment specific evaluation examples.

The remaining chapters were written by teachers at all levels from various part of the country and are divided into three sections. Most of the information is explanatory — here’s how to do an activity — with an inspirational tone — here’s how my students were changed by the experience.
Elementary and Middle School – In this section we learn about an elementary writing workshop for struggling readers that emphasizes the social nature of writing by having the students create web pages about topics of their choice (vampires); a fourth grade collaborative writing project where students create podcasts in order to gain a better understanding of revision; and sixth graders who create math- and science-based digital picture books to help students actively create information that ordinarily they would just read about.

Secondary Grades – In these examples, all from high school classes, we are shown how students participate in blogging for a New Journalism/Technology course by writing and sharing about their chosen topics; how students interpret poetry in the form of a video as part of the Poetry Fusion project; how seniors display their research paper findings (encompassing information from across the curriculum) in such formats as i-movies and Web pages; and how blogging and podcasting are used in a speech class.

College Years – In this final grouping, we see science writing made stronger using graphics and story boarding; students using Web 2.0 tools to create multimodal documents; and literacy narratives presented through hybrid essays using text and images.

The book is an easy read for teachers looking for innovative activities. One thing the book does well is illustrate how rich the experiences of using technology to teach writing can be. The teachers mentioned over and over that their roles change to becoming more of a coach or mentor than someone who presents information just for testing purposes. All of the chapter authors point to some way that the writing experience is deepened and made more meaningful through the use of technology, including:
• better understanding of process—making choices/revising/planning/delivering
• increased interest and ownership of writing
• more writing overall
• real world applications
• more social and interactive experiences.

In the final chapter, the editors situate the assessment discussion in the book under two categories: “Classroom Based Assessment,” where the need is for new criteria that relate more to composing with technology, which results in criteria from print culture being adapted and teachers recognizing the role of creativity in the use of a tool of choice; and “State Curriculum Standards and Standardized Assessments,” where the teachers were able to apply curriculum frameworks, but had more difficulty relating to what standardized testing can and seeks to measure. Authors Reed and Hicks, referring to work by Lankshear & Knobel (2006), provide one example of the difficulties faced: “Much of what we see in the enactment of this curriculum (as a result of the tests) does not engender the kind of changed mindset that a new literacies perspective requires: one that is characterized by openness, collaboration, collective intelligence, distributed authority, and social relations” (p. 136). In their chapter, Frost, Myatt, and Smith also talked about having to deal with “departmental outcomes that still refer to the number of pages students should write.” They felt a sense of “discomfort” when thinking about how to assess differently for “projects that couldn’t be word-count quantified” (p. 182). Many of the authors discuss how the use of technology is seen as an add-on in most standards lists, but the new literacies that students engage in, or assessment frameworks for these new literacies, aren’t included.

This collection presents a good starting point for understanding the need for new and different assessment as we experience the different writing experiences afforded with ever advancing technology possibilities.

Part III:  Review of Norbert Elliot’s and Les Perelman’s (Eds.)  Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White

 

Elliot, N., & Perelman, L. (Eds.) (2012).  Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White.  New York, NY:  Hampton Press.

By Jessica Nastal, University of Wisconsin-Milwaukee
This is the third review in a series of five about Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White, edited by Norbert Elliot and Les Perelman.  The collection is a “testament to White’s ability to work across disciplinary boundaries” as it includes contributions from the writing studies (including the National Writing Project, writing centers, classroom instruction, and writing programs) and educational measurement communities (p. 2).  It is also a snapshot – or a series of snapshots, since it is over 500 pages – of contemporary interests in and concerns about writing assessment; an update on Writing Assessment: Politics, Policies,Practices (1996), edited by White, William Lutz, and Sandra Kamusikiri.

Each chapter in Part III, “Consequence in Contemporary Writing Assessment:  Impact as Arbiter,” drives toward the last sentence of the last chapter in the section, written by Liz Hamp-Lyons:  “You cannot build a sturdy house with only one brick” (p.395).  Elliot and Perelman highlight the section’s dedication to the question of agency, in Edward M. White’s words as the “rediscovery of the functioning human being behind the text” (qtd. p. 371).  I also see the authors in Part III as demonstrating their dedication to understanding the variety of methods and interpretations and social consequences of writing assessment.

Elbow pauses in his “Good Enough Evaluation” and writes, “I seem to be on the brink of saying what any good postmodern theorist would say: there is no such thing as fairness; let’s stop pretending we can have it or even try for it” (p. 305).  He doesn’t cross that brink, of course, and the writers in this section discuss how writing assessment in the twenty-first century might strive for building sturdy houses with many bricks of various shapes and sizes.

In Chapter 17, Peter Elbow urges teachers and administrators of writing to consider “good enough evaluation,” not as a way to get us off the hook of careful evaluation, but as a way to rediscover the human being both writing and reading the text.  In the spirit of White’s practical and realistic forty-year approach, Elbow reminds us that the “value of writing is necessarily value for readers”; and yes, this even means teachers of writing (p. 310). He concludes by explaining that using such evaluation could result in evaluation sessions with “no pretense at ‘training’ or ‘calibrating’ [readers] to make them ignore their own values” (p. 321).

Elliot and Perelman have set up another interesting contrast in Part III:  While many readers will agree with Elbow (how can we not?!), we might have some questions about how this good enough evaluation works in practice, which Doug Baldwin helps to highlight.  How is it that the results become “more trustworthy” through this process (p. 319)?  What makes Directed Self Placement the “most elegant and easy” alternative to placement testing (p. 317; Royer and Gilles discuss the public and private implications of DSP in Chapter 20)?  What impact would multidimensional grading grids, instead of GPAs, have on reading student transcripts (pp. 316-317)?  Baldwin helps to ask how we can ensure the “technical quality” of Elbow’s ideal – though non-standardized – evaluations (p. 327).

For Baldwin, fairness, a concept authors of this section are dedicated to, “refers to assessment procedures that measure the same thing for all test-takers regardless of their membership in an identified subgroup” (p. 328).  He uses the chapter to expose instances that might display “face fairness” – allowing students to choose their prompt, use a computer, or use a dictionary – but that might reveal deeper unfairness for students.  Baldwin’s conclusion provides guidance for those of us concerned about the state of writing and writing assessment in the twenty first century, our diverse populations of students, and our “concerns about superimposing one culture’s definition of ‘good writing’ onto another culture’” (p. 336).

Asao B. Inoue and Mya Poe (Chapter 19), Gita DasBender (Chapter 21), and Liz Hamp-Lyons (Chapter 22) continue probing questions of agency, fairness, and local contexts.  The “generation 1.5” students DasBender worked with were confident in their literacy skills, identified as being highly motivated, and expressed satisfaction with their writing courses.  On the surface, it seemed like the mainstream writing courses served them well; however, instructors believed students “struggled to succeed” in them (p. 376).  DasBender observed, “generation 1.5 students’ self-perceptions as reflected in their DSP literacy profile…is at odds with” the abilities they demonstrate in mainstream writing courses (p. 383).

This conflict seems representative of some of the concerns about contemporary writing assessment in action.  What are programs to do when they employ theoretically sound, fair policies designed to enable student participation and responsibility (“asking them where they fit,” in Royer and Gilles’ words) but that seem to fail in the eyes of instructors or administrators?  DasBender, Elbow, Baldwin, Inoue, Poe, Royer, Gilles, and Hamp-Lyons remind us that while Writing Assessment in the 21^st Century does much to situate writing assessment and Ed White’s role within it, we have more work to do on behalf of all our students – which Part IV:  “Toward a Valid Future” alludes to.

 

NCTE just released a statement about the use of automated essay scoring (AES) in writing assessment. The statement explains why AES shouldn’t be used for evaluating student writing, offers some alternatives, and includes an annotated bibliography of research of machine scoring of student writing. The bibliography is based on the JWA bibliography compiled by Haswell, Donnelly, Hester, O’Neill and Schendel, published in 2012.

What do you think of NCTE’s statement on machine scoring? How can it be useful? Does it go far enough? Is it solidly grounded in research? Let us know what you think.

Will you be attending the Research Network Forum prior to the 2013 CCCC Annual Convention?

If so, stop by the Journal of Writing Assessment‘s table during the Editor’s Roundtable session from 1:15-2:15 PM on Wednesday, March 13, 2013!  For more information about the RNF, visit their website.

We would love to talk with you about potential projects!

See you there!

Part II:   Review of Norbert Elliot’s and Les Perelman’s (Eds). Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White

Elliot, N., & Perelman, L. (Eds.) (2012).  Writing Assessment in the 21st Century:  Essays in Honor of Edward M. White.  New York, NY:  Hampton Press.

By Jessica Nastal, University of Wisconsin-Milwaukee

Norbert Elliot and Les Perleman’s subtitle for their introduction to Part II of Writing Assessment in the 21st Century: Essays in Honor of Edward M. White is “Bridging the Two Cultures,” specifically focusing on the cultures of the academic writing assessment community and the corporate educational community. As Ed White’s career illustrates, groundbreaking work can be done when we do bridge the seemingly vast divide. This section, however, makes it abundantly clear that the bridge assessment leaders advocate for will be difficult to build — especially in the midst of the “corporatization” of universities and writing assessment.[1]

The editors highlight how this section tackles “what may realistically be expected as we balance standards for good assessment practices…with the limits of available resources” as each author offers “practical alternatives, informed research design, and an advanced understanding of the construct of writing and of what is required to improve instructional practice” (p. 150). Central to Part II is recognition that writing instruction and assessment are inextricably linked, which is central to White’s body of work. As Irving Peckham reflects on attitudes toward writing assessment in California in the 1970s, he remarks, “I remember the argument at the time as being cost and efficiency against validity and consequence” (p.170). The authors of Section II echo Peckham’s sentiment as they explore academic and corporate discussions about assessment to provide assessment alternatives that address cost effectiveness, rather than simply cost-efficiency, to improve instruction (see William Condon, Chapter 13),

Contrastingly, however, is Jill Burstein of ETS’s chapter, “Fostering Best Practices in Writing Assessment and Instruction with E-rater.” Burstein appears to employ Lee Odell’s idea of the “given-new” exchange (see Chapter 16) by appealing to scholar-teachers’ dedication to the assessment loop and to generative reflection on our practices. Unfortunately, it reads like a chapter from ETS’s powerful marketing department, trying to sell readers on how E-rater and Criterion are “useful, helpful, and dependable,” in White’s words (this fact and these words are mentioned around 20 times in the 12-page chapter). Burstein claims the automated essay scoring technology “has the potential to enhance and support the writing experience of this large, and culturally and linguistically diverse population” (p. 205). She concludes the chapter by explaining how the programs can support teachers and instruction, not replace them, but this chapter leaves me questioning whether a cultural bridge can be built between corporate educational community and academic writing assessment community.

But as every other chapter makes clear, writing assessment — and White’s work — is built on a dedication to “better fit with our theories and values of language, reading, writing, research, and pluralist democracy” (Broad, p. 261), not to reduce writing to 9 characteristics of grammar, usage, and mechanics. I heed White’s call to work beyond our disciplinary boundaries, but I am wary. I wonder about the reach of ETS and its fellow organizations as their “cost-effective” practices marginalize multilingual and low-income students (see Anne Herrington & Charles Moran, Chapter 12). I am concerned about the increasing push for computer-based writing assessment and how that depersonalizes the acts of writing, interpretation, and communication. Finally, I don’t believe automated essay scoring provides “meaningful and consistent feedback” (p. 204), at least in its current state. Considering writing and assessment in such a limited way undermines, in Bob Broad’s language, “what we really value” about writing assessment.

As Broad and Diane Kelly-Riley (Chapter 8) demonstrate — and as Elliot and Perelman begin the collection — the academic writing assessment community is driven by our collaborations on local, regional, and national levels. It’s what motivates dynamic criteria mapping; what helped Washington State University develop a model, cross-campus portfolio assessment system and Colorado School of Mines engage in critical reflection about their interdisciplinary first-year courses; what encourages us to consider the roles of placement, curricula, response, online instruction, and the constantly-evolving world of writing. The conversations we have as a community, on the WPA-L and in departmental meetings about student writing, are essential to improve teaching and learning in theoretically and ethically sound ways.

 

Writing Assessment and Race Studies Sub Specie Aeternitatis: A Response to Race and Writing Assessment

By Rich Haswell, Haas Professor of English Emeritus, Texas A&M University, Corpus Christi

If understanding is impossible, knowing is imperative.

—Primo Levi

I was filling up my car when I noticed my traveling companion staring at a man in the next row of pumps, staring with fixed malevolence. The man was a stranger. He was also black. With alarm I realized my friend was executing the Southern hate stare. I had never witnessed a hate stare before, only having read about it in John Howard Griffin’s book Black Like Me. I was dumbfounded. We were in the middle of Wyoming, and my companion, a generation older than me, had lived nowhere east of Great Basin country. I had never seen him betray a shred of racism before. The stranger studiously disregarded the stare, finished filling up his car, and drove away. We drove away. Appropriately enough, the station we left was that cultural mix of neon, bad food, fuel, tourists, and truck drivers called Little America.

This incident happened thirty-seven years ago. It was the kind of racism—hateful, publicly hostile, of a kin with sunset laws and burning crosses—that we all pray a larger America has left behind. Racism itself, of course, we have not left behind. It has just diversified. A “new racism” is still present in a legion of forms: structural racism, institutionalized racism, benevolent racism, color-blind racism, scientific racism, culturalist racism, internalized racism, ethnopluralist racism, whitely racism, everyday racism, microaggressive racism—racisms more or less covert, more or less hateful. These terms are the fruit of the new race studies, which have emerged hand in hand with the new racism. Race studies study race in order to understand it better, in order to find ways to deal with people of all races with sympathy and equity, in order, eventually, it hopes, to eliminate racial inequalities everywhere. Equality, of course, may not exactly fit some of the ways that the study of race applies its findings—think, for instance, of affirmative action. So recently Florida’s state board of education set grade-level test goals in reading at 90% for Asian students, 88% for whites, 81% for Hispanics, and 74% for blacks. The members of the Florida board of education do not hate Asian students in setting their particular bar so high but rather hope that whites, Hispanics, and blacks, given time, will catch up.

In doing so, of course, the board affirmed the existence of race. Herein lies a major contradiction in the new race studies. Often they start with critique (“critical race theory”) that argues “race” is a figment, a social, political, and ideological construction, and then they end by affirming and even celebrating racial categories and groups. A common line of argument first deconstructs “race,” then attacks “racism,” and then confirms the equality of all races.

The contradiction runs through Race and Writing Assessment, edited by Asao B. Inoue and Mya Poe (Peter Lang, 2012). At the top of the book, Inoue and Poe state the current truism that “race” is “artificial,” not biological but rather a “social and political construction” (4). And indeed the book tends not to use the terms “racism” and “racist” at all,  preferring expressions such as “racialism,” “racialization,” “raciology,” and “racial formation” (exceptions are chapters by Valerie Balester, Nicholas Behm & Keith D. Miller, and Rachel Lewis Ketai). Yet over and over, as the book furthers projects and programs in writing assessment, it treats race as an objective reality, just as Florida’s board of education has done. In fact an early chapter in the book asks that registrars of USA universities record student’s race (Diane Kelly-Riley), and the last chapter wishes that France’s laws forbidding categorization by racial groups be abeyed for academic scholars (Élizabeth Bautier & Christiane Donahue). Despite Inoue’s start, nowhere afterward does the book deconstruct race. Puzzled, I ask a transgressive question. Does Race and Writing Assessment, in its commendable efforts to act with more fairness toward students of every color, help maintain racism? Let me state my question as a logical aporia. People cannot go about eliminating racism without constructing the notion of race, and the construction of race can only further racism.

This book made me realize again that today, for all of us, “race” is aporetic. It is so by its unavoidable nature as interim, by its being for the time being. One day—a future against which currently many groups fervently fight—interracial marriage globally will eliminate “race.” Eliminated will be the unavoidable first thought on meeting a stranger: “white,” “non-white,” “Asian,” “Amerindian,” “Pacific Islander.” Eliminated will be those euphemistic replacements for “race” that attract scholars in race studies: “population,” “people,” “ethnicity,” “color,” “diversity,” “differentiation,” “nonmainstream,” “minority.” Sub specie aeternitatis, so to speak, the subspecies, or more accurately, the varieties of Homo sapiens today called human “races,” will be no more. Color and pallor will blend into one. But until that phenotypic dispersal, we live racial aporias.

So to the point. Until then, any writing assessment shaped by anti-racism will still be racism or, if that term affronts, will be stuck in racial contradictions. Here are four racial aporias, subsets of the basic aporia expressed above, currently embedded in writing assessment and illustrated by Race and Writing Assessment, although not explicitly expressed there.

To correct racially aligned outcomes, writing assessment must apply benchmarks that are racially aligned. As a whole this book supports the notion that to “democratize” our classrooms (Nicole M. Merola, p.165), no racial group there should be “overrepresented” or “underrepresented” (Inoue, p. 84). Stratification by race, however, is not as straightforward as it appears. In 2007 at Juilliard, with student placement by SAT scores and educational background, enrollment into Fundamentals, the basic writing course, was 50% Asians and 14% whites. In 2010, after a change to an impromptu essay for placement, enrollment in Fundamentals was 41% Asian and 52% white (Anthony Lioi, p. 160). Lioi judges the change “more valid” because now the course population is “more representative of the general racial composition of the student body” (p. 161). This kind of benchmarking Lioi’s co-author Nicole M. Merola calls “isomorphism with the demographics” (p. 164). I won’t examine the unexamined assumption underlying this approach to validation of writing-assessment practices, that the percent of problem writers is the same in every racial group. But why the particular demographic chosen? Why not racial makeup of students applying, or of students graduating, or of USA population as a whole? More to the point, note that this racial isomorphism undercuts other kinds of isomorphism. Why isn’t Juilliard’s basic-writing placement system validated by isomorphic representation of social class, or nationality, or chosen academic major? There is some tacit discrimination going on. Why the effort to represent race evenly and not gender, for instance? Study after study has shown adolescent males performing worse than females on tests of verbal ability, and across the nation males are overrepresented in basic writing courses, yet I know of no professional interest in altering tests or writing placement systems to correct this misrepresentation. Naturally this book aligns itself with race. But as the poet Ai said, “The insistence that one must align oneself with this or that race is basically racist” (1980, p. 277).

Writing-assessment categorization by race erases the individual, yet it is only the individual that can erase race. As Ai continues, “And the notion that without a racial identity a person can’t have any identity perpetuates racism” (p. 277). My emphasis is on a person. Lioi and Merola’s chapter does not sustain a look at any individual student or any individual piece of writing. Nor do the majority of the other chapters (the two exceptions are studies of particular student essays by Anne Herrington & Sarah Stanley, and Zandra L. Jordan). What gets lost when the individual gets lost in discussions of race? For one thing, the notion of race as socially constructed. When “Latino” becomes an assessment category, effaced are persons who have been put in this category but whose individual qualities might well dispute the categorization—e.g., mother of Iberian heritage, father of Amerindian and African heritage. Not only do authors use “Latino” or “Hispanic” as a race category without raising the issue of the legitimacy of that categorization, I don’t remember one author in this collection even raising the issue of “mixed race.” It seems that when “racial formation” becomes the focus, race gets affirmed and the possible elimination of race—phenotypic dispersal—gets tabled. Ai is 1/2 Japanese, 1/8 Choctaw, 1/4 African American, and 1/16 Irish. She is the future, but she doesn’t fit in this book. In Florida’s new reading standards, which set a passing rate of 74% for blacks and 90% for Asian students, what would they do with Ai, or with any student who also happens to be part African and part Asian? The issue marks one spot where race studies in writing assessment need badly to catch up. Women’s studies have been exploring ways out of the trap of essentialism for three decades now.

Writing research into racial formations disregards individual agency but individual agency fuels racial formations. Sartre recounts a woman who hated Jews because a Jewish furrier had ruined one of her furs. Sartre famously asks why she chose to hate Jews and not furriers. What is usually not cited is Sartre’s next sentence: “Why Jews or furriers rather than such and such a Jew or such and such a furrier?” His answer is that she has a “passion,” a “predisposition toward anti-Semitism” (1943, pp. 11-12). In a word Sartre is describing prejudice, the individual, psychological dynamic that the old race studies have investigated for a century now and that still helps drive racial discrimination.  Race and Writing Assessment studiously ignores this dynamic. Inoue and Poe make quite clear why.  “Racial formation,” they say, is “not about prejudice, personal biases, or intent” but about “forces in history, local society, schools, and projects—such as writing programs” (p. 6). The new racism is perpetrated not by individuals but by institutions such as language, curricula, or assessment systems. Inoue and Poe’s contributors agree. They investigate how race is written into machine-scoring programs (Herrington & Stanley), how writing rubrics are mono-cultural (Balester), how a grading contract system favors certain races (Inoue), how African American English is viewed at historically black colleges (Jordan) and at traditionally Anglo graduate schools (Behm & Miller), how directed self-placement systems (Ketai) and standardized placement testing (Lioi & Merola, and Kathleen Blake Yancey) may further racism, and how topics in French school assessment disadvantage immigrant students (Bautier & Donahue). This is all good, in part because it provides a new understanding of racism as embedded and hidden in verbal and social structures.

Yet the erasure of classic hostile individual prejudice—and I can’t think of one specific example in this book—can’t be all good, too. Surely acts of individual racial prejudice have not entirely disappeared from the college writing-assessment scene. Kelly-Riley remarks that in essay-evaluation sessions, “Faculty raters or the other members of the rating community may unwittingly introduce silently held, negative beliefs” (p. 33). I wouldn’t put it so gently. It wasn’t that long ago that Jan and I recorded a composition teacher saying about the student author of an anonymous essay, “he might be a black student and is not probably used to looking at abstractions” (Haswell and Haswell, 1995, p. 245). More to the point, the department-sanctioned rubric, or the grading contract, or the directed self-placement instructions, or the computerized diagnostic program was written by individuals and is applied, essay by essay, by individual teachers and students. Should we let off the hook the individual agency that is still necessary for institutional racism to operate?

Writing scholars position themselves outside institutional racism to understand it but their understanding concludes that there is no outside. By virtue of their scholarly perspective, can writing scholars also be let off the hook? Nowhere in Race and Writing Assessment does any contributor note the possible contradiction between their opposition to institutionalized racism and their belief that institutionalized racism is everywhere—a contradiction, by the way, that has been fully explored in sociological race studies (see Robert Miles, 1989 and Howard Winant, 2005). Behm and Miller talk of the “ubiquity of racism and the hidden power relations that perpetuate it” (p. 135). Ketai assures us that “Race is woven throughout the fabric of placement testing and through conceptions of literacy and educational identity” (p. 145). None of them voice the possibility that this pervasiveness of racial formations might include their own relations, conceptions, and identities. Behm and Miller propose that students can extract themselves through critique (“critical race narratives”) and Ketai offers contextualization as a way to rescue directed self-placement. But they don’t entertain the high probability that critique and context will remain racialized. Nor do the editors note that their book, which repeatedly castigates the stylistic criterion of high academic English as a racial formation, is entirely written in high academic English. And far beyond the margins of Race and Writing Assessment lies the troubling contradiction, often expressed in the literature of racism (“The horror, the horror”), that immersion in the disease of racial inequality risks contamination.

Logically there is no escape from an aporia. As in the hermeneutic circle, where knowledge of the parts is defined by the whole and knowledge of the whole is determined by the parts, in scholarly circles knowledge of institutionalized racism is held by members of the racialized institution. Racial aporias will end only when race itself ends. Primo Levi’s readers sometimes asked him if he understood the level of racial hatred that created the Holocaust. Still remembered is his astonishing response. “No, I don’t understand it nor should you understand it,” he wrote, “it’s a sacred duty not to understand” (1965, p. 227). For to understand is to subsume.

Less remembered, however, is Levi’s continuation a page later: “We cannot understand it, but we can and must understand from where it springs, and must be on our guard. If understanding is impossible, knowing is imperative, because what happened could happen again” (p. 228). By “knowing” Levi meant “modest and less exciting truths, those one acquires painfully, little by little and without shortcuts, with study, discussion, and reasoning, those that can be verified and demonstrated” (p. 229). Levi is asking his readers not to forget that understanding may happen for once and forever but knowledge comes at different times and at different stages for different purposes.

This is why, despite what my own readers may be thinking, I applaud Race and Writing Assessment. Until race disappears, racism and racial formations will be with us, but in the interim, “little by little,” they can be exposed and ameliorated. In this book essay after essay ferrets out unfair writing-assessment practices. That takes courage, especially since some of the practices—such as writing rubrics, grading contracts, computerized evaluation, and directed self-placement—wield a hefty amount of professional esteem. And essay after essay shows more equitable outcomes when particular assessment practices are changed and applied. Scholarly study of racial effects have made the placement systems demonstrably less unfair at Washington State University (Kelly-Riley), the Juilliard School (Lioi), the Rhode Island School of Design (Merola), and Oregon State University (Yancey)—and study of teacher bias at the University of California, Merced and Fayette State University of North Carolina surely will improve teacher practice at those places in the future (Judy Fower & Robert Ochsner). This scholarship may involve Levi’s “modest and less exciting truths” and as I argue it may not have entirely extricated itself from race, but for certain individual students it has made writing assessment more just. During the millennia that it will take for human race to disappear, growth in racial justice is not impossible.

Let’s face it, though. In writing assessment, that growth will happen by short, incremental steps. As Chris Anson’s chapter makes abundantly clear, past compositionists have had an inclination to pretend their operations are free of race, constructed or not. As for future writing-assessment studies, the scholarly stare is hardly comparable to the Southern hate stare, but scholars must, as Levi cautions, constantly be on their guard. Kelly-Riley puts the situation honestly and exactly: “if classrooms are microcosms of our larger society—complete with problems of injustice and inequity—then it is not reasonable to think that all students or teachers or disciplines can be safeguarded against intentional or unintentional bias” (p. 32). But some can, as this book shows. This is the reason to congratulate the editors and contributors of Race and Writing Assessment.

References

Ai. (1980). Ai[Florence Anthony]. In F. C. Locher (Ed.), Contemporary authors. Detroit, MI: Gale Research.

Haswell, J., & Haswell, R. H. (1995). Gendership and the miswriting of students. College Composition and Communication, 46.2, 223-254.

Levi, P. (1965). The Awakening. Boston, MA: Little, Brown.

Miles, R. (1989). Racism. London: Routledge & Kegan Paul.

Sartre, J-P. (1948/1943). Anti-Semite and Jew: An exploration of the etiology of hate. New York: Schocken Books.

Winant, H. (2005). Race and racism: Overview.  In M. C. Horowitz (Ed.), New dictionary of the history of ideas. Vol. 5. Detroit, MI: Charles Scribner’s Sons.