Comparing core concept categorisation models in geo-analytic questions

Abstract

Current question answering (QA) systems lack the ability to provide answers to geo-analytical questions. Geo-analytical questions must be interpreted to know what relevant data and geographical tools require to be used to provide an answer. This study focused on core concept categorisation, which is the first step in developing the aforementioned system. Named-entity recognition, in combination with transformer-based models BERT and RoBERTa, is applied to categorise core concepts in geo-analytical questions. Synonym replacement, a simple data augmentation technique, is applied to overcome data scarcity and results of both models are compared. RoBERTa has a better performance on the original data set and BERT has a better performance on the augmented data set. Both models presented significant improvements when applying synonym replacement. Results of this study can be applied to further develop a geo-analytical QA system.