Assessing different speech interactions in an oral proficiency test

Abstract

This study investigates two aspects of oral proficiency testing of English as a Foreign Language: tasks employed in eliciting language samples and rating scales used to evaluate the elicited samples. It is known that elicitation techniques influence both the quality and quantity of an elicited sample and that rating scales may influence the reliability, validity, and fairness of a learner's assessment. The instrument developed to perform the study is a multi-component test of oral proficiency entitled COAST (Cyprus Oral Academic Skills Test). The test battery consists of four different speechinteractions: Group Discussion, Oral Report, Oral Interview, and Role Play. The test focus is to determine if different speech interactions elicit different kinds of language that could give a broader, more comprehensive picture of a speaker's ability. The Bachman and Palmer Scale (1983) was the instrument chosen to assess the elicited language samples. The scale, which is analytic and based on a theory of communicative competence, is designed to be sensitive to features of linguistic, pragmatic, and sociolinguistic competence. This study investigates whether the chosen scale could differentiate between different degrees of proficiency of the subjects as manifested in the four speech interactions. Finally, the study discusses whether the test battery can serve as a basis for the development of an oral proficiency test for the educational system of Cyprus, which currently lacks an oral test. The study results show that the language elicited by each test component was not perceived by raters to differ across the four speech interactions. The study also demonstrates that the major problems concerning a potential introduction or adaptation of the instrument by the Cypriot educational system pertain to issues of practicality, i.e.adequate tester and rater training. Finally, the study offers recommendations for the development of scales that include features addressing distinctive characteristics of each speech interaction. It is suggested that such scales will be able to differentiate between the subjects' degrees of proficiency in different speech interactions,a beneficial feature for test taker