Description
This study examines the reliability of automatic semantic frame annotations of texts for corpus approaches to discourse analysis and explores how such annotations facilitate the analysis of epistemic positioning.Frame Semantics models how words evoke structured categories of experience represented in semantic frames (Fillmore, 1982). For example, words such as claim, explain and say evoke a Statement frame, while words such as approve, blame and deplore evoke a Judgement frame. Semantic frames are further specified by frame elements such as such as the Speaker and Message in the Statement frame, or the Cognizer and Evaluee in the Judgement frame. Frame-semantic annotation thus identifies rich layers of meaning that enables corpus investigations of discourse features such as attribution (how the speaker/writer arranges viewpoints) and evaluation (how the speaker/writer expresses attitudes and opinions).
Advances in frame-semantic parsing – the task of automatically labelling texts for semantic frames and their frame elements – has made it increasingly feasible to tag large amounts of texts with rich semantic information. However, little is known about the reliability of these tools when applied to annotate texts beyond FrameNet test data. Moreover, while recent studies have explored event representation in news discourse using frame-semantic parsing (e.g. Coschignano et al., 2023; Minnema, 2024), the full potential of this approach for broader discourse studies remains underexplored.
The present study contributes to filling this gap by evaluating two recent open-source frame-semantic parsers – LOME (Xia et al., 2021) and the Semantic Frame Transformer (Chanin, 2023). Their outputs were tested against a set of manual annotations on 300 sentences sampled from UK parliamentary briefing papers on immigration. Results show that both parsers perform less reliably than their reported scores on FrameNet test data. Apart from reporting overall precision and recall, I examine variation in performance across sentence lengths, different part-of-speech categories and discourse-analytical categories. Results highlight both the challenges and opportunities of applying frame-semantic parsers in discourse-oriented corpus research.
To address the challenges identified, I propose several practical strategies for enhancing the usefulness of automatically generated annotations. Among these, pre-processing sentences into smaller syntactic units emerges as the most effective method for improving the recall for frame identification. Other strategies, such as selective post-editing and combining multiple tools, also show promise.
Finally, I demonstrate how patterns of semantic frames provide entry points to examine viewpoint arrangement and evaluation with respect to social actors and social policies in parliamentary briefing papers on immigration. I discuss how frame-semantic parsing – despite current limitations – offers a scalable and systematic resource for the study of epistemic positioning in discourse.
| Period | 11/12/2025 |
|---|---|
| Event title | 4th International Conference for Young Researchers in Cognitive Linguistics |
| Event type | Conference |
| Degree of Recognition | International |