11 Automatically detecting directives with SPICE Ireland
-
Gerold Schneider
Abstract
The pragmatically annotated corpus of spoken Irish English, SPICE Ireland, offers the possibility to explore, analyze, or, as our contribution does, train systems to automatically detect, directives in English. In this study, we evaluate the automatic classification and compare directives in Irish English with directives in British English by using lexical signals in the data sets. To do so, we apply and evaluate two approaches from machine learning, document classification with logistic regression, and deep learning with fastText. Both approaches reach a similar, satisfactory, performance on the task of classifying previously unseen sentences as directive or non-directive: up to 90.5% accuracy, and up to 74.2% Kappa. The reported features deliver a large inventory of indicators for speech acts, such as please indicating imperative or what for interrogatives, but also more fine-grained indicators, such as wait and you know. The results suggest that the Irish English data contains significantly more directives than the British English data, except in formal contexts, but this may be affected by the strong bias of our automatic classification. Our error analysis shows that implicit directives are missed more often, indicating that contextual, social, situational or prosodic knowledge is vital for a small number of the instances, yet our evaluations indicate that classification performance is similar on Irish and British data.
Abstract
The pragmatically annotated corpus of spoken Irish English, SPICE Ireland, offers the possibility to explore, analyze, or, as our contribution does, train systems to automatically detect, directives in English. In this study, we evaluate the automatic classification and compare directives in Irish English with directives in British English by using lexical signals in the data sets. To do so, we apply and evaluate two approaches from machine learning, document classification with logistic regression, and deep learning with fastText. Both approaches reach a similar, satisfactory, performance on the task of classifying previously unseen sentences as directive or non-directive: up to 90.5% accuracy, and up to 74.2% Kappa. The reported features deliver a large inventory of indicators for speech acts, such as please indicating imperative or what for interrogatives, but also more fine-grained indicators, such as wait and you know. The results suggest that the Irish English data contains significantly more directives than the British English data, except in formal contexts, but this may be affected by the strong bias of our automatic classification. Our error analysis shows that implicit directives are missed more often, indicating that contextual, social, situational or prosodic knowledge is vital for a small number of the instances, yet our evaluations indicate that classification performance is similar on Irish and British data.
Kapitel in diesem Buch
- Frontmatter I
- Contents V
- 1 An introduction to sociopragmatic variation 1
-
Part 1: Investigating sociolinguistic variables
- 2 The sociopragmatics of now in corpora of Irish English and Scottish English 11
- 3 Lookit – the story of a pragmatic marker in Irish English 31
- 4 ‘Er, yeah, no, bummer’: An exploration of the ‘new’ discourse pragmatic marker Yeah, No in contemporary Irish English fiction 45
- 5 New speakers of Irish English: Pragmatic and sociophonetic perspectives 69
- 6 Pragmatic markers in Ulster Irish and Irish English 87
- 7 Boring much? Semantic determinants of constructional attraction in Irish English 107
-
Part 2: Metapragmatic structures: Identities, styles, and media
- 8 ‘Oh wait and I tell you . . .’: Narratives, pragmatics, and style in ICE-Ireland 133
- 9 Irish identities in a fictional TV series: Mediatised performance of Derry English in Derry Girls 155
- 10 Salutation and leavetaking formulae in 18th-century varieties of English 173
- 11 Automatically detecting directives with SPICE Ireland 205
- 12 Concluding remarks and future directions in studies on sociopragmatic variation 235
- Index 241
Kapitel in diesem Buch
- Frontmatter I
- Contents V
- 1 An introduction to sociopragmatic variation 1
-
Part 1: Investigating sociolinguistic variables
- 2 The sociopragmatics of now in corpora of Irish English and Scottish English 11
- 3 Lookit – the story of a pragmatic marker in Irish English 31
- 4 ‘Er, yeah, no, bummer’: An exploration of the ‘new’ discourse pragmatic marker Yeah, No in contemporary Irish English fiction 45
- 5 New speakers of Irish English: Pragmatic and sociophonetic perspectives 69
- 6 Pragmatic markers in Ulster Irish and Irish English 87
- 7 Boring much? Semantic determinants of constructional attraction in Irish English 107
-
Part 2: Metapragmatic structures: Identities, styles, and media
- 8 ‘Oh wait and I tell you . . .’: Narratives, pragmatics, and style in ICE-Ireland 133
- 9 Irish identities in a fictional TV series: Mediatised performance of Derry English in Derry Girls 155
- 10 Salutation and leavetaking formulae in 18th-century varieties of English 173
- 11 Automatically detecting directives with SPICE Ireland 205
- 12 Concluding remarks and future directions in studies on sociopragmatic variation 235
- Index 241
