Creation of an Evaluation Corpus and Baseline Evaluation Scores for Welsh Text Summarisation

Mahmoud El-Haj; Ignatius Ezeani; Jonathan Morris; Dawn Knight

Creation of an Evaluation Corpus and Baseline Evaluation Scores for Welsh Text Summarisation

Mahmoud El-Haj
, Ignatius Ezeani
, Jonathan Morris
, Dawn Knight

Computing and Communications

University of Cardiff
Cardiff University

Research output: Contribution to conference - Without ISBN/ISSN › Conference paper › peer-review

Abstract

As part of the effort to increase the availability of Welsh digital technology, this paper introduces the first human vs metrics Welsh summarisation evaluation results and dataset, which we provide freely for research purposes to help advance the work on Welsh summarisation. The system summaries were created using an extractive graph-based Welsh summariser. The system summaries were evaluated by both human and a range of ROUGE metric variants (e.g. ROUGE 1, 2, L and SU4). The summaries and evaluation results will serve as benchmarks for the development of summarisers and evaluation metrics in other minority language contexts.

Original language	English
Pages	14-21
Number of pages	8
Publication status	Published - 15/06/2022
Event	The 4th Celtic Language Technology Workshop (CLTW 2022) - Palais du Pharo, Marseille, France Duration: 20/06/2022 → 20/06/2022 Conference number: 4 http://techiaith.bangor.ac.uk/ticeltaidd/

Workshop

Workshop	The 4th Celtic Language Technology Workshop (CLTW 2022)
Abbreviated title	CLTW 2022
Country/Territory	France
City	Marseille
Period	20/06/22 → 20/06/22
Internet address	http://techiaith.bangor.ac.uk/ticeltaidd/

Access to Document

http://www.lrec-conf.org/proceedings/lrec2022/workshops/CLTW4/pdf/2022.cltw4-1.3.pdfLicence: CC BY: Creative Commons Attribution 4.0 International License
https://aclanthology.org/2022.cltw-1.3/Licence: CC BY: Creative Commons Attribution 4.0 International License

ACC: Welsh Summary Creator
El-Haj, M. (Principal Investigator), Knight, D. (Principal Investigator), Morris, J. (Co-Investigator) & Ezeani, I. (Researcher)
1/05/21 → 1/05/22
Project: Research
DSI: Welsh Automatic Text Summarisation
El-Haj, M. (Principal Investigator)
Welsh Government
1/05/21 → 30/09/22
Project: Research

Welsh Text Summarisation Dataset
Ezeani, I. (Creator), El-Haj, M. (Creator), Morris, J. (Creator) & Knight, D. (Creator), Github, 2022
https://github.com/UCREL/welsh-summarization-dataset
Dataset

Cite this

@conference{92aec905bea54a298e771162e022d480,

title = "Creation of an Evaluation Corpus and Baseline Evaluation Scores for Welsh Text Summarisation",

abstract = "As part of the effort to increase the availability of Welsh digital technology, this paper introduces the first human vs metrics Welsh summarisation evaluation results and dataset, which we provide freely for research purposes to help advance the work on Welsh summarisation. The system summaries were created using an extractive graph-based Welsh summariser. The system summaries were evaluated by both human and a range of ROUGE metric variants (e.g. ROUGE 1, 2, L and SU4). The summaries and evaluation results will serve as benchmarks for the development of summarisers and evaluation metrics in other minority language contexts.",

author = "Mahmoud El-Haj and Ignatius Ezeani and Jonathan Morris and Dawn Knight",

year = "2022",

month = jun,

day = "15",

language = "English",

pages = "14--21",

note = "The 4th Celtic Language Technology Workshop (CLTW 2022), CLTW 2022 ; Conference date: 20-06-2022 Through 20-06-2022",

url = "http://techiaith.bangor.ac.uk/ticeltaidd/",

}

TY - CONF

T1 - Creation of an Evaluation Corpus and Baseline Evaluation Scores for Welsh Text Summarisation

AU - El-Haj, Mahmoud

AU - Ezeani, Ignatius

AU - Morris, Jonathan

AU - Knight, Dawn

N1 - Conference code: 4

PY - 2022/6/15

Y1 - 2022/6/15

N2 - As part of the effort to increase the availability of Welsh digital technology, this paper introduces the first human vs metrics Welsh summarisation evaluation results and dataset, which we provide freely for research purposes to help advance the work on Welsh summarisation. The system summaries were created using an extractive graph-based Welsh summariser. The system summaries were evaluated by both human and a range of ROUGE metric variants (e.g. ROUGE 1, 2, L and SU4). The summaries and evaluation results will serve as benchmarks for the development of summarisers and evaluation metrics in other minority language contexts.

AB - As part of the effort to increase the availability of Welsh digital technology, this paper introduces the first human vs metrics Welsh summarisation evaluation results and dataset, which we provide freely for research purposes to help advance the work on Welsh summarisation. The system summaries were created using an extractive graph-based Welsh summariser. The system summaries were evaluated by both human and a range of ROUGE metric variants (e.g. ROUGE 1, 2, L and SU4). The summaries and evaluation results will serve as benchmarks for the development of summarisers and evaluation metrics in other minority language contexts.

M3 - Conference paper

SP - 14

EP - 21

T2 - The 4th Celtic Language Technology Workshop (CLTW 2022)

Y2 - 20 June 2022 through 20 June 2022

ER -

Creation of an Evaluation Corpus and Baseline Evaluation Scores for Welsh Text Summarisation

Abstract

Workshop

Access to Document

Projects

ACC: Welsh Summary Creator

DSI: Welsh Automatic Text Summarisation

Datasets

Welsh Text Summarisation Dataset

Cite this