Volume 16, Issue 1

This paper highlights areas of concern in the assessment of pragmatics, with the intent of stimulating fresh thinking about the assessment of pragmatics both for research purposes and as a part of classroom instruction. It starts by considering what aspects of ability in pragmatics to assess, and then contrasts the trade-off between the feasibility of obtaining data and the ultimate importance of the data. Next, the conspicuous lack of assessment of ability in L2 pragmatics in language classes is noted. Then follow sections on topics all relating primarily to the assessment of pragmatics for research purposes – the use of mixed methods, data elicitation procedures, and norms used in determining the appropriateness of any given performance in pragmatics. The last two topics deal, respectively, with the perceived relevance of the given assessment by the learners and with the value of collecting verbal report data from the respondents as a means for validating the assessment measures. Finally, considerations regarding the most prominent of these issues are provided.

The assessment of pragmatic skills in a foreign or second language (L2) is usually investigated with regard to language learners, but rarely with regard to non-native language instructors, who are simultaneously teachers and (advanced) learners of the L2. With regard to English as the target language, this is a true research gap, as nonnative English-speaking teachers (non-NESTs) constitute the majority of English teachers world-wide (Kamhi-Stein 2016). Addressing this research gap, this paper presents a modified replication of Bardovi-Harlig and Dörnyei’s (1998) renowned study on grammatical vs. pragmatic awareness, carried out with non-NEST candidates. While the original study asked the participants for a global indication of (in)appropriateness/ (in)correctness and to rate its severity, the participants in the present study were asked to identify the nature of the violation and to suggest a repair. Inspired by Pfingsthorn and Flöck (2017), the data was analyzed by means of Signal Detection Theory with regard to Hits, Misses, False Alarms and Correct Rejections to gain more detailed insights into the participants’ metalinguistic perceptions. In addition, the study investigated the rate of successful repairs, showing that correct problem identification cannot necessarily be equated with adequate repair abilities. Implications for research, language teaching and language teacher education are derived.

The present study investigates variability among raters from different linguistic backgrounds, who evaluated the pragmatic performance of English language learners with varying native languages (L1s) by using both self- and peer-assessments. To this end, written discourse completion task (WDCT) samples of requesting speech acts from 10 participants were collected. Thereafter, the participants were asked to assess their peers’ WDCTs before assessing their own samples using the same rating scale. The raters were further asked to provide an explanation for their rating decisions. Findings indicate that there may indeed be a link between a rater’s language background and their scoring patterns, although the results regarding peer- and self-assessment are mixed. There are both similarities and differences in the participants’ use of pragmatic norms and social rules in evaluating appropriateness.

The study of pragmatic competence has gained increasing importance within second language assessment over the last three decades. However, its study in L2 language testing is still scarce. The aim of this paper is to research the extent to which pragmatic competence as defined by the Common European Framework of Reference for Languages (CEFR) has been accommodated in the task descriptions and rating scales of two of the most popular Oral Proficiency Interviews (OPIs) at a C1 level: Cambridge’s Certificate in Advanced English (CAE) and Trinity’s Integrated Skills in English (ISE) III. To carry out this research, OPI tests are first defined, highlighting their differences from L2 pragmatic tests. After pragmatic competence in the CEFR is examined, focusing on the updates in the new descriptors, CAE and ISE III formats, structure and task characteristics are compared, showing that, while the formats and some characteristics are found to differ, the structures and task types are comparable. Finally, we systematically analyse CEFR pragmatic competence in the task skills and rating scale descriptors of both OPIs. The findings show that the task descriptions incorporate mostly aspects of discourse and design competence. Additionally, we find that each OPI is seen to prioritise different aspects of pragmatic competence within their rating scale, with CAE focusing mostly on discourse competence and fluency, and ISE III on functional competence. Our study shows that the tests fail to fully accommodate all aspects of pragmatic competence in the task skills and rating scales, although the aspects they do incorporate follow the CEFR descriptors on pragmatic competence. It also reveals a mismatch between the task competences being tested and the rating scale. To conclude, some research lines are proposed.

This qualitative classroom study investigated the development of pragmatic competence in academic discussions through content analysis of student reflective writing. The aims of the study were: to understand the greatest challenges that students faced during the learning process, the causes of those challenges, and the most successful strategies that students employed to overcome the challenges. In addition, the analysis investigated other significant themes in the reflective writing that related to the students’ experiences in developing their pragmatic competence in discussions. Five advanced English for Academic Purposes (EAP) students at a Sino-US institution in China participated over the course of a semester. Results showed that common challenges included: hesitation resulting in missed opportunities to speak, lack of clarity when speaking, inability to repair communication breakdowns, and difficulty with listening comprehension. Self-reflection allowed the learners to understand the various reasons for the challenges they faced and to develop appropriate pragmalinguistic and sociopragmatic strategies for coping with them. It also enabled the instructor to make suggestions suited to learners’ specific needs. In addition to revealing specific challenges, causes, and strategies that students employed, themes that emerged through content analysis included the impact of students’ emotional lives on their learning and performance, as well as the value of authentic communication in the development of pragmatic competence for academic discussions. This exploratory classroom investigation provides suggestions for teaching pragmatic competence in academic discussions and for additional classroom explorations that empower learners to develop autonomy.

This paper investigates the evaluation of ritual frame indicating expressions (RFIEs) in two groups of L2 learners: British English learners of Chinese and Mainland Chinese learners of English. RFIEs are expressions by means of which speakers confirm their awareness of rights and obligations in a particular standard situation. Previous research in applied linguistics has largely ignored the production and evaluation of such forms, despite the fact that they are pragmatically-loaded and, as such, are very important for the development of the (meta)pragmatic competence of L2 learners. In our study, we have conducted an experiment in which we exposed our learners to both authentic and constructed uses of RFIEs. The evaluation of their (meta)pragmatic competence has revealed notable linguacultural differences between the two groups of L2 learners.