JUCS - Journal of Universal Computer Science 21(4): 561-586, doi: 10.3217/jucs-021-04-0561
Sentiment and Behaviour Annotation in a Corpus of Dialogue Summaries
Norton Trevisan Roman‡, Paul Piwek§, Ariadne Maria Brito Rizzoni Carvalho|, Alexandre Rossi Alvares‡‡ University of São Paulo, São Paulo, Brazil§ The Open University, Milton Keynes, United Kingdom| University of Campinas, Campinas, Brazil
Corresponding author: Norton Roman ( norton@usp.br ) © Norton Roman, Paul Piwek, Ariadne Maria Brito Rizzoni Carvalho, Alexandre Alvares. This article is freely available under the J.UCS Open Content License. Citation:
Roman NT, Piwek P, Ariadne Maria Brito Rizzoni Carvalho, Alvares AR (2015) Sentiment and Behaviour Annotation in a Corpus of Dialogue Summaries. JUCS - Journal of Universal Computer Science 21(4): 561-586. https://doi.org/10.3217/jucs-021-04-0561 |  |
Abstract
This paper proposes a scheme for sentiment annotation. We show how the task can be made tractable by focusing on one of the many aspects of sentiment: sentiment as it is recorded in behaviour reports of people and their interactions. Together with a number of measures for supporting the reliable application of the scheme, this allows us to obtain sufficient to good agreement scores (in terms of Krippendorf's alpha) on three key dimensions: polarity, evaluated party and type of clause. Evaluation of the scheme is carried out through the annotation of an existing corpus of dialogue summaries (in English and Portuguese) by nine annotators. Our contribution to the field is twofold: (i) a reliable multi-dimensional annotation scheme for sentiment in behaviour reports; and (ii) an annotated corpus that was used for testing the reliability of the scheme and which is made available to the research community.
Keywords
corpus annotation, automatic dialogue summarisation, natural language processing, computational linguistics, sentiment analysis