Conversing with oral history archives: A study into the scope of an AI-driven chatbot aiming for optimal knowledge discovery

Abstract

Digital archives are capable of housing vast amounts of data which raises the importance of effective navigation to support knowledge discovery. Traditional navigation techniques, such as search bars and filters often fall short in maximizing knowledge discovery which is especially important in complex data collections like oral history archives. AI-driven chatbots offer a promising alternative due to their ability to understand context, learn from user queries, and provide interactive feedback. However, research on AI chatbots as a navigation tool remains limited. This study explores the impact of chatbot answering strategies on knowledge discovery and tries to answer the question: Should an AI-driven chatbot provide direct answers or is a guidance-based approach more effective in facilitating knowledge discovery? An experiment was conducted in which 31 participants completed a set of tasks using either a direct-answering chatbot or a guiding chatbot. Task completion time, perceived usability and knowledge discovery were measured among other metrics. Mann-Whitney U tests and Spearman’s rank correlation analyses revealed that users of the direct-answering chatbot scored better on knowledge discovery, knowledge retention as well as on perceived usability. These findings contradict initial expectations but align with cognitive theories on information-seeking behavior. These theories suggest that users prefer the most efficient path to the most relevant information retrieval. Further research is needed to validate these results and explore the broader implications for AI-driven navigation in digital archives.