Applying Chatbot models in natural language processing to an intelligent query system for reference document retrieval

Abstract

    In the era of increasingly vast, diverse, and rapidly evolving data, the demand for accurate and reliable information retrieval has become more critical than ever. Traditional search systems, which primarily rely on keywords and metadata, often fail to understand contextual nuances, leading to imprecise results and a lack of credible references. These limitations reduce the reliability of retrieved information and hinder effective analysis and decision-making. To overcome these challenges, the DataChatBot system was developed, leveraging Large language models (LLMs) in combination with advanced techniques such as external information retrieval and vector databases. As a result, the system enables information to be stored and searched as embedded vectors, enhancing semantic search capabilities, supporting logical reasoning, and delivering accurate, source-cited responses. This approach not only improves the relevance and trustworthiness of search results but also empowers users to verify information with ease, thereby optimizing the processes of information discovery, analysis, and decision-making.