Enriching training data with syntactic knowledge and the effect on performance of a neural network on natural language processing tasks

Abstract

Compared to neural networks (NN), humans can learn new concepts using only very little data. The ability to learn so efficiently might be due to the use of ab- stractions. To find similarities between human and machine learning this research will analyze if NN benefits from syntactic information during training. We will aim to answer the following question: How does enriching training data with syntactic knowledge affect the performance of a NN on natural language processing tasks? This research examines the results of Long Short Term Memory models (LSTM) trained on two different types of datasets; one without Part of Speech tags (a form of abstract knowledge) and a dataset that is supplemented with POS-tags. The results show that an LSTM trained on a relatively small dataset supplemented with POS- tags outperforms an LSTM trained on a regular dataset. The increase in performance might suggest that neural networks benefit from abstract information, which in turn might show some similarities in the way humans and machines learn.