Sighted MicroPsi Agents in Minecraft: Object Recognition Using Neural Transfer Learning and Automated Dataset Collection

Abstract

We need intelligent agents that interact with an open world through perception and action. Although previous research has investigated how cognitive architectures could be interfaced with virtual worlds, it is not yet known if and how we could build an interface with a virtual world that meets the characteristics for suitable Artificial General Intelligence (AGI) research environments. Vision is arguably our primary source of information. It is though unclear, how we can provide an agent with vision without world-specific annoted data. To address hese challenges, the Malmo API for Minecraft is connected to the MicroPsi cognitive archi- ecture, which I have tested against AGI virtual world requirements laid out by AI researchers. n order to bootstrap vision, transfer learning is applied on automatically extracted data from an experimental setup in Malmo. Agents can use this network to recognise objects in-game n real time, with high accuracy. This shows that networks trained outside of virtual worlds can generalise to in-game objects, when they are provided with a few in-game examples. This provides a basis for research on intelligent agents in virtual worlds.