Using peer assessment in online courses: a qualitative approach to deciding which behavioural and structural variables to incorporate in MOOC design

Abstract

Extensive evaluation of MOOCs can be nearly impossible for teachers due to the immense amount of work. The most applied evaluation tools in MOOCS therefore comes in the form of peer assessment. This form of assessment is affected by a multitude of variables, which have not always been accounted for in existing research. The present study aims to provide a clear overview of knowledge and gaps on the different variables that affect peer assessment, by categorizing these into behavioural and structural variables. Specifically, a lack of knowledge exists on the behavioural variables affecting the peer assessment process and outcomes. This knowledge is extended by empirical findings. The present study consists of two parts with two different research methods: the first part provides a structured overview of variables that affect peer assessment and identifies knowledge gaps through a systematic literature review. To generate new insights on most particularly the behavioural variables, three focus group sessions with university students were carried out. During these focus groups, opinions and experiences with regard to peer assessment were discussed. These focus group sessions comprise the second part of this research. This paper especially contributes to existing knowledge by unravelling the variable motivation and the potential ways to increase student motivations for thoroughly and truthfully engaging in the peer assessment process. Although this study contributes to determining how peer assessment is most optimally implemented, many ambiguities still exist around the optimal implementation of certain variables. Therefore, providing a straightforward advice for optimal course design is not reasonable, though a clear overview with recommendations and trade-offs for each variable is given. The present study helps course designers to decide which identified variables of peer assessment to implement in the course, and remaining knowledge gaps are identified for which recommendations for future research are made.