December 2011 Volume 7 Issue 4
Spring Edition
PDF Full Version


Title
A Study of the Effects of Raters’ Background Knowledge and Raters’ Training in Rating Thai Pilots’ English-Speaking Proficiency

Author
Sutas Dejkunjorn and Kanchana Prapphal
Chulalongkorn University, Thailand

Biodata
Sutas Dejkunjorn, M.A. is a Ph.D. candidate in Language Assessment and Evaluation, English as an International Language (EIL) Program, Chulalongkorn University. He is also an expert in the field of aviation since he is a captain and an aviation English instructor working for Thai Airways International PLC.

Former director of CULI, Professor Kanchana Prapphal was appointed Professor Emeritus by Chulalongkorn University Council on March 31, 2011. Professor Kanchana graduated from the Faculty of Arts, Chulalongkorn University. She received her M.A. from the University of Hawaii and Ph.D. from the University of New Mexico. She was President of Thailand TESOL from 1986-1987, CULI Director from 1976-1999, and Director of Academic Testing Center of Chulalongkorn University from 2000-2008. She has contributed in the areas of language testing and evaluation, English teaching and English as an International Language.

Abstract
The purpose of this research is to study the effects of the raters’ background knowledge and raters’ training in their ratings when they assess Thai pilots’ English speaking proficiency. Three speech samples were presented to 20 raters for their assessment. The raters were divided into four groups, namely trained linguistic raters, untrained linguistic raters, trained operational raters, and untrained operational raters.  Each of the raters assessed and provided scores for each speech sample. The assessment was conducted using the International Civil Aviation Organization (ICAO) rating scale and descriptors. The score results were analyzed using 2x2 ANOVA to examine relationships between the raters’ background knowledge and raters’ training.  Results showed that the raters’ background knowledge and raters’ training did not affect their decision making, in both main and interaction effects. However, the factor of training rather than background seemed to affect the scores.