Skip Navigation



Applied Linguistics Advance Access published online on November 24, 2009
Applied Linguistics, doi:10.1093/applin/amp040
This Article
Right arrow Full Text
Right arrow Full Text (PDF)
Right arrow Alert me when this article is cited
Right arrow Alert me if a correction is posted
Services
Right arrow Email this article to a friend
Right arrow Similar articles in this journal
Right arrow Alert me to new issues of the journal
Right arrow Add to My Personal Archive
Right arrow Download to citation manager
Right arrowRequest Permissions
Google Scholar
Right arrow Articles by Lee, Y.-W.
Right arrow Articles by Kantor, R.
Social Bookmarking
Add to CiteULike   Add to Connotea   Add to Del.icio.us  
What's this?

© Oxford University Press 2009

Toward Automated Multi-trait Scoring of Essays: Investigating Links among Holistic, Analytic, and Text Feature Scores

Yong-Won Lee1, Claudia Gentile2 and Robert Kantor3

1Seoul National University, 2Mathematica Policy Research, and 3Educational Testing Service


  Abstract

The main purpose of the study was to investigate the distinctness and reliability of analytic (or multi-trait) rating dimensions and their relationships to holistic scores and e-rater® essay feature variables in the context of the TOEFL® computer-based test (TOEFL CBT) writing assessment. Data analyzed in the study were holistic and multi-trait essay scores provided by human raters and essay feature variable scores computed by e-rater® (version 2.0) for two TOEFL CBT writing prompts. It was found that (i) all of the six multi-trait scores were not only correlated among themselves but also correlated with the holistic score, (ii) high correlations obtained among holistic and multi-trait scores were largely attributable to the impact of essay length on both holistic and multi-trait scoring, and (iii) some strong associations were confirmed between several e-rater variables and multi-trait rating dimensions. Implications are discussed for improving the multi-trait scoring of essays, refining e-rater essay feature variables, and validating automated essay scores.

Received for publication 1 August 2009.
Add to CiteULike CiteULike   Add to Connotea Connotea   Add to Del.icio.us Del.icio.us    What's this?




Disclaimer: Please note that abstracts for content published before 1996 were created through digital scanning and may therefore not exactly replicate the text of the original print issues. All efforts have been made to ensure accuracy, but the Publisher will not be held responsible for any remaining inaccuracies. If you require any further clarification, please contact our Customer Services Department.