Ming Huei Lin (University of Birmingham)
Evaluation of writing by computerized measurements and human raters -A pilot study
With the assistance of information technology, more researchers than ever are showing interest in observing student writing performance using word count, token-type ration (TTR), and T-units (e. g., Fellner and Apple, 2005; Jarvis, 2002; Lin, 2007; Morris and Cobb, 2004). It seems assumed by these researchers that these numbers will serve as an indicator of writing development for students (McNeill, 2006). Surprisingly, however, McNeill’s survey on studies examining the correlation between computing measurements and scores given by human raters revealed that word count (or tokens), outperforming TTR and T-units, is significantly correlated with holistic scores given by readers (p. 39). It is more than difficult to simply take these numbers as potential evidence of the process in students’ writing ability. The researcher, therefore, intends to re-examine McNeill’s observation by investigating a currently compiled mini-corpus of 198 pieces of online EFL journal writing of 40,000 words produced by Taiwanese EFL majors. In addition, this researcher aims to look into the correlation between students’ collocation performance and the scores generated by human readers, as this is rarely discussed. The above correlation will be examined employing SPSS; the computerized measurements will be carried out using Antconc3.2.1.