Investigating The Generalization Ability Of Convolutional Neural Networks For Interpreted Languages

Abstract

In this study the generalization capacity of Convolutional Neural Networks (CNNs) for interpreted languages is investigated. Two CNN models, one of which included a curriculum, are trained on two interpreted languages of different complexity. The results show that a CNN, contrary to previous findings for Long-Short-Term-Memory Networks, does not benefit from a curriculum during training. Performance of models on the more complex interpreted language shows adequate generalization ability, while performance on the less complex language shows no generalization ability at all. This suggests that a CNN prefers complex training data over less complex training data, for it forces the model to capture more generally applicable features from which it benefits during testing. Overall the reported results of this study show that CNNs possess a generalization capacity for interpreted languages that is competitive with Recurrent and Recursive models from the literature.