Online learning modes can provide convenience and flexibility to students. But communicating the value of online education in technical and professional communication should not end there. Program directors should rearticulate the narrative about the value of online graduate education beyond flexibility and convenience by reevaluating the ways that program assessment is designed and implemented. This pilot study suggests that a community of inquiry framework can help to communicate the value of the online learning environment to a variety of stakeholders, including prospective and current students, administrators, instructors, and potential employers.

This article explores the dynamic practice of inviting community members to grade college students on their work in community-engaged partnerships. The authors articulate theories of writing assessment with theories of reciprocity to argue that community-based student evaluations can be a valid and ethical form of assessment, and discuss a case study in which local youth graded college students to offer eight best practices for implementing community-based assessment. As reciprocity is often underemphasized in practice, community evaluations provide a strategy for shifting power toward community members, potentially reinvigorating applications of reciprocity to make them more substantial and meaningful.

This case study reports on the experiences of designing and assessing the effectiveness of a faculty development program on writing across the curriculum (WAC). The report focuses on the question: What are the key components of an effective faculty development program to integrate WAC into engineering and scientific courses taught by faculty in those disciplines? Situating the case: Two main models of WAC implementation exist: direct instruction, which uses writing specialists to deliver instruction to engineering and science students, and the department-centered model, which instructs faculty in engineering and scientific disciplines to teach writing as part of technical courses. How the case was studied: A report of the experiences of the authors and the feedback from the participants. About the case: The workshop was aimed at teachers in various disciplines and covered these main topics: fundamentals of writing theory and pedagogy, writing assignment design and assessment, and situating writing assignments in courses across the disciplinary curriculum. It took place over 10 weeks during a 15-week semester and included large- and small-group meetings, consultations with the members of the university WAC program, and peer review of writing assignment drafts. Conclusions, limitations, and suggestions for future research: Key challenges in developing the workshop included designing ways to bridge the conceptual gap between the participants' and WAC instructors' understanding of the role of writing in disciplinary courses, limited time available to the participants, and scheduling challenges. The workshop was given seven times. Most of the faculty participants (90%) generally found it to be very effective or effective. Studies of workshops with larger populations of trainees are suggested.

Proponents of reframing argue that prophetic pragmatism entails redirecting contemporary education reforms. While this judgment may defend our professional standing, it overlooks the consequences of redirecting reform's appeals to global competition, which preclude public participation in defining the goals and measures of literacy education. This article forwards an alternate pragmatism for attending to the public consequences of reform discourse.

In this article, the author proposes that the emergence of digital, disease-tracking applications over the past ten years like HealthMap (healthmap.org) and Flu Near You (flunearyou.org) that allow non-experts to contribute information about emergent public health threats have facilitated a "do-it-yourself (DIY)" risk assessment ethic. Focusing in particular on Flu Near You (FNY), a crowdsourced, flu-tracking program, the author argues that some participants use the mapping feature to curate their own risk information experience in determining the preventative behaviors they may want to engage in (if any) to prevent flu. As outbreaks of infectious diseases increase (Smith et al., 2014), mHealth technologies like disease-tracking apps are evolving as an important risk assessment tool for both public health experts as well as non-expert, public audiences. Better understanding how non-experts use such information can inform not only the design of these apps but visual risk communication strategies more generally speaking.

New York: Oxford University Press, pp. 356 ISBN 978-0-19-982728-2

This essay offers an early assessment, after the first 100 days, of Donald Trump’s bewildering ascendency to the US presidency. It examines his apocalyptic rhetoric as a spectacle of salvation by demolition and deal making, a polarizing and demonizing politics that trades in deception and distraction. The spectacle, whether it is a means to an end or an end in itself, functions to distort democratic politics and to displace public dissent over the negative impact of economic globalization. The question raised is whether and how dissent might be channeled more constructively through a narrative of fairness that balances interests equitably and deliberates policy options credibly.

This book examines some of the many struggles that various programs in the field have endured in light of the "Great Recession" and shares the faculty’s initiative to take action to ensure the sustainability of their programs. Eleven essays written by 20 different contributors from a range of institutions assist in successfully achieving the book’s purpose—namely, providing methods and models for how academic programs can “do more with less” in an age of austerity. Program coordinators and those who teach program assessment should take heart in knowing that they do not face these challenges alone, and they might use this book to brainstorm and identify solutions for their programs.

Background: This paper examines creativity in content management (CM) by presenting a case study analysis of an original, open-source, Web-based learning-management system (LMS). It explores existing literature and suggests new ideas about creativity and CM. Research questions: How can we conceptualize creativity in relation to CM? How can creativity be operationalized into specific design practices within CM? What dimensions of creativity should be considered when designing or evaluating creative CM systems?Situating the case: Theoretical research in content-management systems (CMSs) and LMSs has revealed a gap in our field's current understanding of how creativity relates to CM. Research studies related to the measurement of creativity have provided insight into characteristics and methodologies that could be adopted and adapted to evaluate creativity in CM. Studies from the learning sciences have investigated where LMSs fall short and suggested where new opportunities exist to better facilitate the informational needs of users. These works have pointed to a need for research on integrating creativity and CM, both in content and in the systems that manage content, and have laid the groundwork for this study. Methodology: This research investigates theories of creativity as they relate to CM by conducting a case study analysis of unique instructional software designed to deliver content to students enrolled in a university course. The primary evidence is taken from notes about the design and evaluation of the software and from survey data illustrating students' user experiences. About the case: We developed a classification strategy for exploring creativity along three dimensions-importance, novelty, and affect-and then used this strategy to explore a unique system's approach to deal with content challenges in each of those areas. User experience impressions provide evidence of successes and failures of experimental CM in these domains. Conclusions: The study finds that this original LMS design did contain features that exhibited novelty, importance, and affect, and that such features can be used to identify creativity in product design as well as to assess the design of complex software systems such as CMSs and LMSs. This detailed analysis of an original design for an LMS suggests new ideas for implementing and using CMSs and LMSs in technical communication. The study concludes by presenting a rubric for evaluating creativity in CMSs and LMSs, or for designing such systems with creativity in mind.

370 RHETORICA Graban finds that she is unable to delineate Gougar's affiliations as stable and permanent because her relationships with other suffragists and politicians evolved throughout her life. And lastly, class-consciousness as the organizing topoi allows Graban to "complicate the language surrounding ... the middle class lens [typically used] to view social uplift in Gougar's work" (p. 154). In her final chapter, Graban presents more textual examples of irony through a critical frame—one from Golda Meir, prime minster of Israel, one from Madeline Albright, American diplomat, and another from Barbara Jordan, investigator of the Watergate Scandal. Although some might think Graban falls into the trap of "tokenism," whereby examples of a few stand in for all women, she works against it as she selects archives based on their ironic potential and qualities. Furthermore these archives are situated panhistorically so as not to essentialize women or their writings as representative of a specific place or time. In addition to alleged "tokenism," some might find fault with the scant textual evidence taken from Anne Askew's archive in chapter one. Yet, these critics should keep in mind the erasure of women's rhetoric throughout the Renaissance and employ their critical imagination to reconsider the potential for the evidence that does exist.2 It is also important to note that Graban not only examines textual evidence, she also employs "historical residue" as evidence—residue that includes: organizing topoi, intersecting contexts, and the positioning of audiences. Graban's scholarship resets the terms of scholarly engagement for those working in the field of rhetoric and history by resituating irony and using it to destabilize historical narratives and the ways in which these nar­ ratives are remembered. Tiffany Kinney, University of Utah, Salt Lake City Laurent Pernot, Epideictic Rhetoric: Questioning the Stakes of Ancient Praise, Austin: University of Texas Press, 2015. xiv, 166 pp. ISBN 978-1-4773-1133-2 In 1993 Pernot's highly acclaimed, two-volume work, La rhétorique de l'éloge dans le monde gréco-romain appeared. In 2012 at the meeting of the Rhetoric Society of America, with ISHR sponsorship, Pernot conducted a three-day seminar on epideictic for twenty participants (among whom was the current reviewer). Using the format of the seminar but drawing content from his earlier book, Pernot has now produced a concise but 2 J. J. Royster and G. Kirsch, Feminist Rhetorical Practices: New Horizons for Rhetoric, Composition and Literacy Studies (Carbondale: Southern Illinois University Press, 2012), 72-73. ' Reviews 371 example-packed history, analytical summary, and contextualizing assessment of the theoretical treatises and actual speeches of ancient Greco-Roman epideictic rhetoric. Two questions drive the presentation: (1) How was it that epi­ deictic, originally the minor player in the famous trio of judicial, deliberative, and epideictic, acquired the far-and-away dominant role of the three in the Imperial age? and (2), What, in fact, was that role? Through an impressive breadth and depth of reading and a precise deployment of select ancient sour­ ces, Pernot shows how "every encomium is at once a literary work, a moral problem, and a social rite" (ix). In Chapter 1, "The Unstoppable Rise of Epideictic" (1-28), Pernot surveys the meager evidence for epideictic texts from Classical Greece to Republican Rome (1-9). Epideictic was, in those centuries, something of a sidecar to the normally stand-alone two wheels of deliberative and judicial oratory. Yet, as the chapter title suggests, the epideictic sidecar will "tri­ umph" (9) in the Imperial period, and the path of that triumph is delineated in the rest of the chapter (9-23). The conclusion? The Imperial period, for the whole of that Greco-Roman world—especially in Greek—"was the begin­ ning of a new rhetorical world order, in which oratory served no longer to rip apart an adversary or to cow an assembly, but to spread honeyed praise and trumpet meritorious conduct with previously unparalleled frequency and variety" (28). Chapter 2, "The Grammar of Praise," (29-65) surveys the methods and means of epideictic in light of the teaching texts that survive, drawing espe­ cially from Menander Rhetor, but Plato, Isocrates, and Aristotle are also quoted and even Aelius...

This study explored the effects of transcription on translation products and processes of adolescent students in grades 4 to 9 with and without persisting specific language disabilities in written language (SLDs-WL). To operationalize transcription ability (handwriting and spelling) and transcription mode (by pen on digital tablet or by standard US keyboard), diagnostic groups contrasting in patterns of transcription ability were compared while composing autobiographical (personal) narratives by handwriting or by keyboarding: Typically developing students (n=15), students with dyslexia (impaired word reading and spelling, n=20), and students with dysgraphia (impaired handwriting, n=19). They were compared on seven outcomes: total words composed, total composing time, words per minute, percent of spelling errors, average length of pauses, average number of pauses per minute, and average length of language bursts. They were also compared on automaticity of transcription modes-writing the alphabet from memory by handwriting or keyboarding (they could look at keys). Mixed ANOVAs yielded main effects for diagnostic group on percent of spelling errors,, words per minute, and length of language burst. Main effects for transcription modes were found for automaticity of writing modes, total words composed, words per minute, and length of language bursts; there were no significant interactions. Regardless of mode, the dyslexia group had more spelling errors, showed a slower rate of composing, and produced shorter language bursts than the typical group. The total number of words, total time composing, words composed per minute, and pauses per minute were greater for keyboarding than handwriting, but length of language bursts was greater for handwriting. Implications of these results for conceptual models of composing and educational assessment practices are discussed.

In recent years, embedding writing into subject teaching through genre-based writing instruction (GBWI) has been advocated in tertiary education. However, little is known about how this approach can be shaped and implemented in this context. In a design-based research study in Dutch higher professional education, we aimed to explore how GBWI can be used to scaffold students’ writing within the subject of Event Organization and to what extent students learned to use the typical features of the genre ‘event proposal’. A 5-week subject-specific writing intervention was designed and subsequently enacted by a subject lecturer in a first-year class involving 13 students. Using a coding scheme for interactional scaffolding strategies, five interaction fragments were analyzed against the background of designed scaffolding and learning goals. The fragments indicated that the interplay of designed scaffolding (instructional materials and activities) and interactional scaffolding (teacher-student interactions) promoted students’ writing performance over time. Comparison of students’ pre- and posttests by means of an analytic scoring scheme pointed to statistically significant growth in the use of typical genre features (d=1.41). Together, the results of this design-based research study indicate the potential of GBWI for scaffolding and promoting tertiary students’ writing.

Free online survey tools provide a practical learning-by-induction platform for business communication instructors interested in trying out an advanced multidisciplinary survey activity coupled with an innovative teaching design. More than just building skills in marketing, survey projects marshal a wider set of thinking and doing activities that build student competency in the interrelated disciplines of communication, consumer analysis, and research. The design and sequence of a survey-learning module are outlined as well as expected learning outcomes, assessment considerations, and suggestions for exploring the interdisciplinary opportunities that surveys afford.

Our aim was to explore the influence of mobile learning on students’ acquisition of conceptual knowledge of business communication, as well as on the development of their communication skills. We compared the performance of three groups of students according to the pedagogical approach that we used with them: a mobile learning group, a conventional group, and a control group. Our findings suggest that a mobile learning intervention leads to an improvement in student performance in a formal assessment and that it will also have a positive impact on learning outcomes.

Text-driven, quantitative methods provide new ways to analyze student writing, by uncovering recurring grammatical features and related stylistic effects that remain tacit to students and those who read and evaluate student writing. To date, however, these methods are rarely used in research on students transitioning into US postsecondary writing, and especially rare are studies of student writing that is already scored according to high-stakes writing expectations. This study offers a corpus-based, comparative analysis of higher- and lower-scoring Advanced Placement (AP) exams in English, revealing statistically significant syntactic patterns that distinguish higher-scoring exams according to “informational production” and lower-scoring essays according to “involved” or “interactional” production (Biber, 1988). These differences contribute to what we label emphatic generality in the lower-scoring essays, in which writers tend to foreground human actors, including themselves. In contrast, patterns in higher-scoring essays achieve what we call elaborated specificity, by focusing on and explicating specific, often abstract, concepts.These findings help uncover what is rewarded (or not) in high-stakes writing assessments and show that some students struggle with register awareness. A related implication, then, is the importance of teaching register awareness to students at the late secondary and early university level—students who are still relative novices, but are being invited to compose informationally dense prose. Such register considerations, and specific features revealed in this study, provide ways to help demystify privileged writing forms for students, particularly students for whom academic writing may seem distant from their own communicative practices and ambitions.

This essay describes a graduate course, The Nineteenth-Century Novel in Context, that I developed and taught in fall 2011 at the Graduate Center, City University of New York. The essay was developed from an oral presentation that was part of a teaching panel at the Northeast Victorian Studies Association annual conference in the spring of 2013. The course was my final effort to “go wide” in teaching Victorian literature in its larger context, a desire that grew increasingly difficult to satisfy as the canon of Victorian literature became enlarged and thus somewhat unstable. I also wanted to organize the readings so that my students might get a sense of the literary context in which Victorian readers might have experienced the individual texts when they read them in the nineteenth century. In an effort to describe how I got to the syllabus for The Nineteenth-Century Novel in Context (included as an appendix), I give a personal sense of the history of the field of Victorian literature over the last fifty years, tracing the development of the field of English literature in general and Victorian literature in particular. I end with my evaluation of the course I developed, its strengths and its weaknesses, and what I learned from it.

This article examines the teaching of a multimodal pedagogy in an online technical communication classroom. Based on the results of an e-portfolio assessment, the authors argue that multimodality can be taught successfully in the online environment if the instructor carefully plans and scaffolds each assignment. Specifically, they argue for an increased emphasis within the technical communication classroom on teaching the e-portfolio as a genre that not only exemplifies students’ multimodal literacies but also establishes their identities as technical communicators in the 21st century. This article provides a model for teaching multimodal composition in the online technical communication classroom and calls for more scholarship on teaching the e-portfolio in the digital environment.

This article reports the background, methods, and results of a 7-year project (2007–2013) that assessed the writing of undergraduate business majors at a business college. It describes specific issues with writing assessment and how this study attempted to overcome them, largely through a situated assessment approach. The authors provide the results of more than 3,700 assessments of nearly 2,000 documents during the course of the study, reporting on scores overall and for each rubric criterion and comparing the scores of English and business assessors. They also investigate how two curricular interventions were evaluated through this assessment project. Although overall, the writing of these business majors was assessed as good, results showed noteworthy differences between the scores of English and business assessors and a noteworthy impact for one of the curricular interventions, an effort to improve the material conditions of writing instruction. The authors conclude by discussing some next steps and implications of this project.

Self-annotation forces students to build sideline commentary for their own writing. As a self-assessment strategy, annotation at every stage of the writing process turns underprepared writers into more confident decision makers and communicators.

Lloyd Bitzer's passing came as deeply sad news. He was an exceptional person in all respects. I was fortunate to have been his student at the University of Wisconsin–Madison and to have experienced Lloyd in my life as a mentor, a colleague in the discipline, a confidant, a friend, and a role model. The discipline of rhetoric was fortunate to have had him among its ranks as a leading theorist. He was among those most responsible for pushing rhetorical studies into new territory during the latter part of the twentieth century. Lloyd was the principle investigator on and driving force behind the National Developmental Project on Rhetoric, which involved forty scholars from philosophy, rhetoric, communication, English, and sociology at the Wingspread and Pheasant Run conferences at the beginning of the 1970s and which culminated in The Prospect of Rhetoric, the volume he coedited with his colleague Edwin Black. And Philosophy and Rhetoric was fortunate to have him grace its pages with his scholarship and editorial advice. His iconic essay “The Rhetorical Situation” inaugurated the journal in 1968 as the lead article. It set the stage for reconsidering rhetoric in terms of its philosophical commitments.Lloyd was not a prolific publisher, but each of his articles were gems of careful scholarship and tight reasoning, and they demonstrate an unfailing sense for ideas that matter and an understanding of the impact those ideas could have on future work. His 1959 Quarterly Journal of Speech article “Aristotle's Enthymeme Revisited” broke new ground by decoupling the form of pisteis Aristotle regarded as the heart of persuasion from its logical form. His 1960 QJS article “A Re-evaluation Campbell's Doctrine of Evidence” argued that Campbell, in following Hume, had inverted the two-millennial-old Western tradition that established reason as the capital of right action and instead located it in the passions. His subsequent editor's introduction to the edited republication of Campbell's Philosophy of Rhetoric and his 1969 Philosophy and Rhetoric article “Hume's Philosophy in George Campbell's Philosophy of Rhetoric” meticulously made the case for Hume's role in introducing rhetoric into the new country wherein its study led to understanding human nature. In 1978, when consideration of the public sphere was just beginning to emerge as a scholarly topic in the literature on rhetoric, Lloyd published his award- winning essay “Rhetoric and Public Knowledge,” in which he considered the necessary conditions for distinguishing between audiences and publics. It was not a coincidence that two years earlier he broke form with the practice of association presidents in the then Speech Communication Association of offering as their presidential address reflections on the discipline when he presented a version of this paper as his presidential address. His choice was an expression of his belief that presidents of scholarly societies should lead by example of their scholarship.Lloyd's presidential address, as much as anything, captured his sense of himself as a scholar and teacher and spoke to what he considered the nobility of his and our work. Studying with him was at once exhilarating, fearsome, calming, and affirming. He was demanding of his students, excited by ideas, not given to tolerating sloppy thinking or unsupported argument, quick to affirm student insights and progress, able to express and inspire confidence in his students' work, and generous with his time and counsel, always willing to assist his students' growth and prosperity. My friend Tom Farrell, another of Lloyd's doctoral students, captured well how lasting an impact our mentor had when, in the prime of our careers, he commented “I still write for Lloyd.” So did I; so do I still.In May 2015, the Rhetoric Society of America held its biennial summer Institute at the University of Wisconsin–Madison. I was filled with anticipation for the event, which is unique in its format and impact on its participants, for being once more in Madison where I had done my doctoral studies, and for the opportunity to spend time with colleagues, former students, and dear friends in the discipline. At the center of my excitement was the dinner date Lloyd and I had arranged. That evening was vintage Lloyd: he and his incomparable spouse Jo Ann arriving precisely on time, dinner at a favorite restaurant, lively and wide-ranging conversation covering shop talk, politics, the university, mutual friends, our children, and grandchildren. Too soon the evening ended, but Lloyd insisted that we should drive to his home outside Madison to drop off Jo Ann and have a nightcap before he took me back to my hotel on campus. He made certain we extended the evening so our conversation might continue. His characteristic care for how our time was spent conveyed more than words the intimacy of personal regard.Lloyd was not comfortable with warm expressions (he edited my dissertation acknowledgment of him, insisting I delete comments on what he meant to me—he meant the world—as something I might find embarrassing for their warmth in later years). But he knew how to convey his warmth and how to acknowledge it in return. He brought me to believe in myself as a young scholar, he filled me with admiration and trust, he inspired delight in intellectual work, and more than anyone he awakened my sense of its essential dignity. He touched the profession and this journal as a scholar. He touched me as a person. I shall remember Lloyd always with affection and gratitude. He enriched my life and I shall miss him dearly.

Mathematics standards in the United States describe communication as an essential part of mathematics. One outlet for communication is writing. To understand the mathematics writing of students, we conducted a synthesis to evaluate empirical research about mathematics writing. We identified 29 studies that included a mathematics-writing assessment, intervention, or survey for students in 1st through 12th grade. All studies were published between 1991 and 2015. The majority of assessments required students to write explanations to mathematical problems, and fewer than half scored student responses according to a rubric. Approximately half of the interventions involved the use of mathematics journals as an outlet for mathematics writing. Few intervention studies provided explicit direction on how to write in mathematics, and a small number of investigations provided statistical evidence of intervention efficacy. From the surveys, the majority of students expressed enjoyment when writing in mathematics settings but teachers reported using mathematics writing rarely. Across studies, findings indicate mathematics writing is used for a variety of purposes, but the quality of the studies is variable and more empirical research is needed.

Studies on writing development have grown in diversity and depth in recent decades, but remain fragmented along lines of theory, method, and age ranges or populations studied. Meaningful, competent writing performances that meet the demands of the moment rely on many kinds of well-practiced and deeply understood capacities working together; however, these capacities’ realization and developmental trajectories can vary from one individual to another. Without an integrated framework to understand lifespan development of writing abilities in its variation, high-stakes decisions about curriculum, instruction, and assessment are often made in unsystematic ways that may fail to support the development they are intended to facilitate; further, research may not consider the range of issues at stake in studying writing in any particular moment.To address this need and synthesize what is known about the various dimensions of writing development at different ages, the coauthors of this essay have engaged in sustained discussion, drawing on a range of theoretical and methodological perspectives. Drawing on research from different disciplinary perspectives, they propose eight principles upon which an account of writing development consistent with research findings could be founded. These principles are proposed as a basis for further lines of inquiry into how writing develops across the lifespan.

This qualitative study reports on teachers’ (formative) feedback practices in writing instruction. Observations and interviews were used to collect data from 10 upper-secondary school teachers of English as a Foreign Language (EFL) writing classes in Norway. The findings indicate that while the teachers attempt to comply with the requirements of the national curriculum regarding formative assessment, and acknowledge the pivotal role of feedback in that pedagogy, the dominant tendency is still to deliver feedback to a finished text. As such, there is limited use of feedback for that text and no resubmission of the text for new assessment, while feedforward is reduced to the correction of language mistakes, which does not foster writing development except for language accuracy. The limited use of formative feedback suggests the need for more systematic professional development of the teachers.

Sustainable feedback practices, that can encourage self-regulation of performance and improvement in future work beyond an immediate task, require our students to be active participants in, and users of, the feedback we provide. Critical to this participation are the internal feedback mechanisms of reflection and self-assessment. They require students to make evaluations about their own writing without the aid of external agents, which in turn can encourage better use of teacher feedback. Moreover, dialogic collaborative feedback that encourages this type of self-evaluation through interactive cover sheets has been featured in existing practitioner research studies. This teaching article presents an extension to the use of such cover sheets to include student self-evaluation and reflection in relation to specific marking criteria as part of an existing feedback cycle on a first-year undergraduate course. Observations from the practitioner research presented here highlight how the inclusion of such rubric criteria not only helped to develop students’ confidence in independently monitoring and evaluating their writing but also heightened awareness of the rhetorical features of their texts.

As part of well-planned formative assessment, feedback can help students to understand the demands of a summative assessment task, evaluate their current level of performance, and then find ways to close the gap. As students take a more active role in this process, their feedback can be thought of as becoming ‘feedforward’ since it serves a specific purpose and drives student action. As the value of formative assessment design is becoming emphasised in higher education, summative assessment practices need to be re-evaluated in terms of how well they support learning as opposed to just supporting valid judgements of student performance. However, despite significant discussion of Assessment for Learning and Learning-Oriented Assessment, resubmission practices are largely overlooked even though resubmission can be a key event in whether students are retained.As part of a learning support department’s effort to provide effective feedback on academic writing, students referred for support were offered two types of feedback: one was simple correction, the other was in-depth dialogic feedback which followed “feedback for learning” guidance (Askew and Lodge 2000). Student engagement with the two types of feedback was analysed by looking at the changes students made to their work and feedback from their subject tutor (including the resubmission grade). The tutor’s feedback was also analysed to see if any intentions for the resubmission task could be inferred.Results suggest that corrective feedback is highly efficient in enabling students to pass resubmissions and that more in-depth feedback is much less efficient. This paper highlights some of the ways in which resubmission practices can unknowingly encourage surface approaches, and suggests some ideas for how learning support can better align with subject tutors to enable resubmission to become more learning-oriented.

Background: A shift of focus has been marked in recent years in the development of automated essay scoring systems (AES) passing from merely assigning a holistic score to an essay to providing constructive feedback over it. Despite all the major advances in the domain, many objections persist concerning their credibility and readiness to replace human scoring in high-stakes writing assessments. The purpose of this study is to shed light on how to build a relatively simple AES system based on five baseline writing features. The study shows that the proposed AES system compares very well with other state-of-the-art systems despite its obvious limitations. Literature Review: In 2012, ASAP (Automated Student Assessment Prize) launched a demonstration to benchmark the performance of state-of-the-art AES systems using eight hand-graded essay datasets originating from state writing assessments. These datasets are still used today to measure the accuracy of new AES systems. Recently, Zupanc and Bosnic (2017) developed and evaluated another state-of-the-art AES system, called SAGE, which enclosed new semantic and consistency features and provided for the first time an automatic semantic feedback. SAGE’s agreement level between machine and human scores for ASAP dataset #8 (the dataset also of interest in this study) was measured and had a quadratic weighted kappa of 0.81, while it ranged for 10 other state-of-the-art systems between 0.60 and 0.73 (Chen et al., 2012; Shermis, 2014). Finally, this section discusses the limitations of AES, which come mainly from its omission to assess higher-order thinking skills that all writing constructs are ultimately designed to assess. Research Questions: The research questions that guide this study are as follows: RQ1: What is the power of the writing analytics tool’s five-variable model (spelling accuracy, grammatical accuracy, semantic similarity, connectivity, lexical diversity) to predict the holistic scores of Grade 10 narrative essays (ASAP dataset #8)? RQ2: What is the agreement level between the computer rater based on the regression model obtained in RQ1 and the human raters who scored the 723 narrative essays written by Grade 10 students (ASAP dataset #8)? Methodology: ASAP dataset #8 was used to train the predictive model of the writing analytics tool introduced in this study. Each essay was graded by two teachers. In case of disagreement between the two raters, the scoring was resolved by a third rater. Basically, essay scores were the weighted sums of four rubric scores. A multiple linear regression analysis was conducted to determine the extent to which a five-variable model (selected from a set of 86 writing features) was effective to predict essay scores. Results: The regression model in this study accounted for 57% of the essay score variability. The correlation (Pearson), the percentage of perfect matches, the percentage of adjacent matches (±2), and the quadratic weighted kappa between the resolved scores and predicted essay scores were 0.76, 10%, 49%, and 0.73, respectively. The results were measured on an integer scale of resolved essay scores between 10-60. Discussion: When measuring the accuracy of an AES system, it is important to take into account several metrics to better understand how predicted essay scores are distributed along the distribution of human scores. Using average ranking over correlation, exact/adjacent agreement, quadratic weighted kappa, and distributional characteristics such as standard deviation and mean, this study’s regression model ranks 4th out of 10 AES systems. Despite its relatively good rank, the predictions of the proposed AES system remain imprecise and do not even look optimal to identify poor-quality essays (binary condition) smaller than or equal to a 65% threshold (71% precision and 92% recall). Conclusions: This study sheds light on the implementation process and the evaluation of a new simple AES system comparable to the state of the art and reveals that the generally obscure state-of-the-art AES system is most likely concerned only with shallow assessment of text production features. Consequently, the authors advocate greater transparency in the development and publication of AES systems. In addition, the relationship between the explanation of essay score variability and the inter-rater agreement level should be further investigated to better represent the changes in terms of level of agreement when a new variable is added to a regression model. This study should also be replicated at a larger scale in several different writing settings for more robust results.

Technique Identification: A new graphical technique is presented for visualizing and assessing inter-rater agreement in discrete ordinal or categorical data, such as rubric ratings.  To that aim, a chance-corrected Kappa with two new features is derived. First, it is based on interpreting ratings for each subject as vectors to visualize the data. This is done by creating two-dimensional vectors from a subject-rating summary table, sorting the vectors by their slopes, and plotting them in that order to create a trajectory that displays all the data in context. Second, it presents a graph and accompanying statistics (Kappa, p -value) for each pair of ratings in an organized display so that all useful comparisons of the data are visually displayed and statistically assessed. This information is presented on a logical grid, usually called facets . Kappa is calculated in the usual way, by referencing the actual results with an average of random rating assignments. This average becomes a reference line on each graph as a visual cue, as well. The statistical basis for the Kappa and significance testing are derived, and the test assumptions are specified. Value Contribution: The most commonly used statistics for inter-rater agreement, such as the Cohen Kappa or Inter-Class Correlation, give only a single parameter estimate of reliability from which to make judgments about ratings data. The technique presented here constructs graphs of all the data that allow visual inspection of the ratings versus a reference curve that represents chance-matching. The detailed reports on inter-rater agreement can show how to fine-tune ratings systems, such as understanding which parts of an ordinal scale are working best. This solves a practical problem for researchers who rely on rating-type classification by revealing which overall aspects of the rating system need to be improved and adds to the list of tools available for assessing rating reliability. In creating this approach to analysis of rater data, human usability is emphasized. Specifically, the use of geometry is designed to facilitate interpretability rather than being a mathematical derivation from first principles. Technique Application: Two applications are given, both involving social meaning-making. The first uses data from wine-judging to illustrate how the method can illuminate expertise in that domain. The results reproduce published findings that were based on a classical statistical method. A second sample application uses data from a university assessment of student writing in which ratings on a developmental scale are assigned by course instructors to their students. The rating program is an example of social meaning-making that can be used to generate larger data sets than are typical for classroom-based assessment programs. The analysis shows the strengths and weaknesses of the rating system in terms of reliability and demonstrates how that knowledge leads to improvements in assessment. Directions for Further Research: An argument is made for a public library of inter-rater data for empirical use by researchers. The social aspects of rating are discussed, and there is an illustration of the potential to derive new measures of inter-rater agreement from the meaning-making program that produces the data.

Background: This study focuses on construct representation and inter-reader agreement and reliability in ePortfolio assessment of 1,315 writing portfolios. These portfolios were submitted by undergraduates enrolled in required writing seminars at the University of Pennsylvania (Penn) in the fall of 2014.  Penn is an Ivy League university with a diverse student population, half of whom identify as students of color. Over half of Penn’s students are women, 12% are international, and 12% are first-generation college students. The students’ portfolios are scored by the instructor and an outside reader drawn from a writing-in-the-disciplines faculty who represent 24 disciplines. The portfolios are the product of a shared curriculum that uses formative assessment and a program-wide multiple-trait rubric. The study contributes to scholarship on the inter-reader reliability and validity of multiple-trait portfolio assessments as well as to recent discussions about reconceptualizing evidence in ePortfolio assessment.  Research Questions: Four questions guided our study: What levels of interrater agreement and reliability can be achieved when assessing complex writing performances that a) contain several different documents to be assessed; b) use a construct-based, multi-trait rubric; c) are designed for formative assessment rather than testing; and d) are rated by a multidisciplinary writing faculty?   What can be learned from assessing agreement and reliability of individual traits? How might these measurements contribute to curriculum design, teacher development, and student learning? How might these findings contribute to research on fairness, reliability, and validity; rubrics; and multidisciplinary writing assessment? Literature Review: There is a long history of empirical work exploring the reliability of scoring highly controlled timed writings, particularly by test measurement specialists. However, until quite recently, there have been few instances of applying empirical assessment techniques to writing portfolios.  Developed by writing theorists, writing portfolios contain multiple documents and genres and are produced and assessed under conditions significantly different from those of timed essay measurement. Interrater reliability can be affected by the different approaches to reading texts depending on the background, training, and goals of the rater. While a few writing theorists question the use of rubrics, most quantitatively based scholarship points to their effectiveness for portfolio assessment and calls into question the meaningfulness of single score holistic grading, whether impressionistic or rubric-based. Increasing attention is being paid to multi-trait rubrics, including, in the field of writing portfolio assessment, the use of robust writing constructs based on psychometrics alongside the more conventional cognitive traits assessed in writing studies, and rubrics that can identify areas of opportunity as well as unfairness in relation to the background of the student or the assessor. Scholars in the emergent field of empirical portfolio assessment in writing advocate the use of reliability as a means to identify fairness and validity and to create great opportunities for portfolios to advance student learning and professional development of faculty.  They also note that while the writing assessment community has paid attention to the work of test measurement practitioners, the reverse has not been the case, and that conversations and collaborations between the two communities are long overdue. Methodology: We used two methods of calculating interrater agreement: absolute and adjacent percentages, and Cohen’s Unweighted Kappa, which calculates the extent to which interrater agreement is an effect of chance or expected outcome. For interrater reliability, we used the Pearson product-moment correlation coefficient. We used SPSS to produce all of the calculations in this study.  Results: Interrater agreement and reliability rates of portfolio scores landed in the medium range of statistical significance.  Combined absolute and adjacent percentages of interrater reliability were above the 90% range recommended; however, absolute agreement was below the 70% ideal.  Furthermore, Cohen’s Unweighted Kappa rates were statistically significant but very low, which may be due to “kappa paradox.” Discussion: The study suggests that a formative, rubric-based approach to ePortfolio assessment that uses disciplinarily diverse raters can achieve medium-level rates of interrater agreement and reliability. It raises the question of the extent to which absolute agreement is a desirable or even relevant goal for authentic feedback processes of a complex set of documents, and in which the aim is to advance student learning. At the same time, our findings point to how agreement and reliability measures can significantly contribute to our assessment process, teacher training, and curriculum. Finally, the study highlights potential concerns about construct validity and rater training.  Conclusion: This study contributes to the emergent field of empirical writing portfolio assessment that calls into question the prevailing standard of reliability built upon timed essay measurement rather than the measurement, conditions, and objectives of complex writing performances.  It also contributes to recent research on multi-trait and discipline-based portfolio assessment.  We point to several directions for further research:  conducting “talk aloud” and recorded sessions with raters to obtain qualitative data on areas of disagreement; expanding the number of constructs assessed; increasing the range and granularity of the numeric scoring scale; and investigating traits that are receiving low interrater reliability scores. We also ask whether absolute agreement might be more useful for writing portfolio assessment than reliability and point to the potential “kappa paradox,” borrowed from the field of medicine, which examines interrater reliability in assessment of rare cases. Kappa paradox might be useful in assessing types of portfolios that are less frequently encountered by faculty readers. These, combined with the identification of jagged profiles and student demographics, hold considerable potential for rethinking how to work with and assess students from a range of backgrounds, preparation, and abilities.  Finally, our findings contribute to a growing effort to understand the role of rater background, particularly disciplinarity, in shaping writing assessment. The goals of our assessment process are to ensure that we are measuring what we intend to measure, specifically those things that students have an equal chance at achieving and that advance student learning.  Our findings suggest that interrater agreement and reliability measures, if thoughtfully approached, will contribute significantly to each of these goals.

Background: Attention Deficit Hyperactivity Disorder (ADHD) is a mental health disorder. People diagnosed with ADHD are often inattentive (have difficulty focusing on a task for a considerable period), overly impulsive (make rash decisions), and are hyperactive (move excessively, often at inappropriate times). ADHD is often diagnosed through psychiatric assessments with additional input from physical/neurological evaluations. Written Language Disorder (WLD) is a learning disorder. People diagnosed with WLD often make multiple spelling, grammar, and punctuation mistakes, have sentences that lack cohesion and topic flow, and have trouble completing written assignments. Typically, WLD is also diagnosed through psychological educational assessments with additional input from physical/neurological evaluation. Literature Review: Previous research has shown a link between ADHD and writing difficulties. Students with ADHD have an increased likelihood of having writing difficulties, and rarely is there a presence of writing difficulties without ADHD or another mental health disorder. However, the presence of writing difficulties does not necessarily indicate the presence of a WLD. There are other physical and behavioral factors of ADHD that can contribute to a student having a WLD as well. Therefore, a statistical association between these factors (in conjunction with written performance) and WLD must first be established. Research Question: To determine the statistical association between WLD and physical and behavioral aspects of ADHD that indicate writing difficulties, this research reviewed methodologies from the literature pertaining to contemporary diagnoses of writing difficulties in ADHD students, and reveal diagnostic methods that explicitly associate the presence of WLD with these writing difficulties among students with ADHD. The results demonstrate the association between writing difficulties and WLD as it pertains to ADHD students using an integrated computational model employed on data from a systematic review. These results will be validated in a future study that will employ the integrated computational model to measure WLD among students with ADHD. Methodology: To measure the association of WLD among students with ADHD, the authors created a novel computational model that integrates the outcomes of common screening methods for WLD (physical questionnaire, behavioral questionnaire, and written performance tasks) with common screening methods for ADHD (physical questionnaire, behavioral questionnaire, adult self-reporting scales, and reaction-based continuous performance tasks (CPTs)). The outcomes of these screening methods were fed into an artificial neural network (ANN ) first, to ‘artificially learn’ about measuring the prevalence of WLD among ADHD students and second, to adjust the prevalence value based on information from different screening methods. This can be considered as the priming of the ANN. The ANN model was then tested with data from previous studies about ADHD students who had writing difficulties. The ANN model was also tested with data from students without ADHD or WLD, to serve as control. Results: The results show that physical, behavioral, and written performance attributes of ADHD students have a high correlation with WLD (r = 0.72 to 0.80) in comparison to control students (r = 0.30 to 0.20), substantiating the link between WLD and ADHD. It should be noted that due to lack of female participation, most studies in the literature only employed and reported on the relationship between WLD and ADHD for male participants. Discussion and Conclusion: By testing ADHD students and control students against the WLD criteria, the study shows a strong correlation between WLD and ADHD. There are limitations to the results’ accuracy in terms of a) sample size (average n=88, mean age = 19, 8 studies used for a meta-analysis), b) analysis (original study reviewing ADHD factors first, WLD factors second), and c) causation (the study only reviews prevalence of WLD in ADHD students, not causation). A clinical trial will validate the data and address some of these limitations in a future phase of the research. A computational causal model will be introduced in the discussion portion to illustrate how causation between writing metrics and WLD as it pertains to ADHD can be achieved. These results open the door to advancing pedagogical techniques in education, where students afflicted with ADHD and/or WLD could not only receive assistance for the behavioral aspects of their disorder, but also expect assistance for the learning aspects of their disorder, empowering them to succeed in their studies.

This essay examines the challenges and opportunities that characterize teaching literature in contemporary high schools and colleges—an educational milieu that has become increasingly dominated by standardized testing, skills assessment, and careerism.

Handwritten texts carry significant information, extending beyond the meaning of their words. Modern neurology, for example, benefits from the interpretation of the graphic features of writing and drawing for the diagnosis and monitoring of diseases and disorders. This article examines how handwriting analysis can be used, and has been used historically, as a methodological tool for the assessment of medical conditions and how this enhances our understanding of historical contexts of writing. We analyze handwritten material, writing tests and letters, from patients in an early 20th-century psychiatric hospital in southern Germany (Irsee/Kaufbeuren). In this institution, early psychiatrists assessed handwriting features, providing us novel insights into the earliest practices of psychiatric handwriting analysis, which can be connected to Berkenkotter’s research on medical admission records. We finally consider the degree to which historical handwriting bears semiotic potential to explain the psychological state and personality of a writer, and how future research in written communication should approach these sources.

Abstract This longitudinal mixed-methods study assesses students’ perceptions of the writing center at a large (approximately 11,325 students) multi‑campus two‑year college. The survey was collaboratively designed, with faculty and student participation; it presents findings from 865 student respondents, collected by peer tutors‑in‑training. The study offers a baseline assessment (Fall 2014) of the writing center, prior to wide-sweeping changes in recruitment, staffing, and training models, as well as a post-assessment (Fall 2015) analysis of the changes in student knowledge of the WC and its purpose. It also offers data on the trajectory of student development in relation to number of sessions attended. In 2014, students’ experiences at the writing center were inconsistent; the poorly articulated mission of the WC adversely affected students’ knowledge scores, and the center’s reliance on editorial-like feedback, given predominately by adjunct faculty, contributed to inconsistent reportage in perceived learning by attended sessions. Many of these trends, however, reversed in 2015. This paper seeks to demonstrate the important role that RAD research can play in evaluating student learning within writing center contexts and articulating how and at what moments, and under what conditions, learning and development occurs in the student-writing center relationship. It also offers a replicable experimental method that researchers at other institutions can adapt and apply to their own institutional contexts and programmatic needs.

Abstract In an attempt to create more meaningful and effective assessment, the Howe Writing Center at Miami University implemented a new post-consultation/exit survey. During the course of the Fall 2012 semester, over 800 students responded to the post-consultation survey. Writing center theory has documented the limitations of the post-consultation survey; however, this type of feedback still represents the best and most accessible way to assess and expand the knowledge of writing centers. This assessment project provided important feedback concerning the writing center at Miami University about student demographics that use the writing center, including academic year and classes students wanted to work on. The assessment project also contributes to writing center theory and discourse by providing a different narrative for non-native English speaking students and native English speaking students that use the writing center. The assessment challenges the view that writing from non-native English speaking students is only concerned with so-called "lower order" writing issues and writing from native English speaking students is primarily concerned with so-called "higher order" writing issues. Instead, it was found that non-native English speaking students are interested in working on many "higher order" concerns and were very similar, after sentence-level concerns, in their writing needs to native English speaking students.

In this program profile, we describe the stretch/studio program recently implemented at the University of New Mexico. This program responds both to an institutional move away from remediation and to the large number of linguistically and racially diverse students at our institution. In this profile, we describe the new program’s curriculum, which focuses on and values the linguistic and cultural diversity of our students. We also share the initial results of our assessment of the program and our plans for the future. We offer this profile as a successful model for adaptation by other writing programs that are also implementing stretch/studio courses and/or that have a growing number of linguistically and culturally diverse students on their campus.

This article describes and theorizes a failed writing program assessment study to question the influence of “the rhetoric of agreement,” or reliability, on writing assessment practice and its prevalence in validating institutional mandated assessments. Offering the phrase “dwelling in disagreement” as a queer perspective, the article draws on expertise theory and notions of ambience and attunement in rhetorical scholarship to illustrate the complexity, unpredictability, and disorder of the teaching and assessment of writing. Adopting a queer sensibility approach, the article marginally disrupts “success” as assumed by order, efficiency, and results in writing assessments and explores how scholars might reimagine ideas, practices, and methods to differently understand a queer rhetoricity of assessment and learning.

This profile presents New Jersey City University’s Writing Assessment Program from its creation in 2002 to its elimination in 2017. The program arose as an attempt to raise the writing skills of the diverse, first-generation teacher certification candidates in the College of Education. Despite political missteps, the program gained greater administrative support in 2009, and in this second stage, the program capitalized on greater institutional support to use data-driven analysis to inform policy. In 2014, however, New Jersey moved to require the Praxis CORE, and the Writing Assessment Program became obsolete. This profile discusses the many ways in which a locally developed, student-centered, and instruction-driven assessment program can raise student skills and the losses involved in a shift from local to national assessment.

To address the needs and interests of primary stakeholders in a writing program, this article presents a model of “grassroots” assessment that involves instructors from all ranks as well as students in the development, facilitation, and interpretation of assessment results. The authors describe two assessment plans that measured student and instructor perceptions about curricular changes, the individual results of those assessments, and the conclusions the authors came to about the importance and challenges of involving a range of stakeholders in assessment.