A UNIFIED RAG AND TOOL-CALLING FRAMEWORK WITH SAFETY AND COMPLIANCE CONTROLS FOR BANKING CHATBOTS IN VIETNAM
Keywords
DOI:
https://doi.org/10.34238/tnu-jst.15914Abstract
Banking chatbot systems in Vietnam face significant challenges when using Large Language Models (LLM) while constantly having to provide accurate information, perform safe service tasks, and follow banking regulations. These requirements become difficult when these systems use the Vietnamese language due to issues such as missing diacritics, unclear word separation, and unconventional query language, leading to low information retrieval efficiency. This paper proposes a collective architecture that integrates RAG and function calling mechanisms through multi-layer security control tools and mechanisms within a single conversation. Our proposed framework models record retrieval and service operations as tools that can be flexibly organized by LLMs. The retrieval module uses a hybrid approach, combining the BM25 mechanism and semantic embedding representation through Reciprocal Rank Fusion and applying Vietnamese-specific preprocessing steps such as normalization, diacritic recovery, and banking terminology mapping. Results from the test dataset show that the hybrid retrieval method is much better than using single methods. Experiments also indicate that removing Vietnamese diacritics significantly decreases retrieval quality while dictionary-based diacritic restoration only provides slight improvement. These results suggest that combining a collective integrated architecture with language-specific processing is crucial for building a practical Vietnamese banking chatbot.Downloads
References
P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela, “Retrieval-augmented generation for knowledge-intensive NLP tasks,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33, pp. 9459-9474, 2020.
V. Karpukhin, B. Oğuz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih, “Dense passage retrieval for open-domain question answering,” in Proc. 2020 Conf. Empirical Methods in Natural Language Processing (EMNLP), Online, 2020, pp. 6769-6781, doi: 10.18653/v1/2020.emnlp-main.550.
L. Gao, X. Ma, J. Lin, and J. Callan, “Precise zero-shot dense retrieval without relevance labels,” in Proc. 61st Annual Meeting of the Association for Computational Linguistics (ACL), Toronto, Canada, 2023, pp. 1762-1777, doi: 10.18653/v1/2023.acl-long.99.
G. V. Cormack, C. L. A. Clarke, and S. Büttcher, “Reciprocal rank fusion outperforms condorcet and individual rank learning methods,” in Proc. 32nd Int. ACM SIGIR Conf. Research and Development in Information Retrieval, Boston, MA, USA, 2009, pp. 758-759, doi: 10.1145/1571941.1572114.
R. Pradeep, R. Nogueira, and J. Lin, “The expandomono-duo design pattern for text ranking with pretrained sequence-to-sequence models,” arXiv:2101.05667, 2021, doi: 10.48550/arXiv.2101.05667.
N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang, “Lost in the middle: How language models use long contexts,” Transactions of the Association for Computational Linguistics, vol. 12, pp. 157-173, 2024, doi:10.1162/tacl_a_00638.
T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” In Proc. 37th International Conference on Neural Information Processing Systems, Dec. 2023, pp. 68539-68551, doi: 10.5555/3666122.3669119.
S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “ReAct: Synergizing reasoning and acting in language models,” in Proc. 11th International Conf. Learning Representations (ICLR), arXiv:2210.03629, 2023, doi: 10.48550/arXiv.2210.03629.
Y. Liu, G. Deng, Y. Li, K. Wang, Z. Wang, X. Wang, T. Zhang, Y. Liu, H. Wang, Y. Zheng, and Y. Liu, “Prompt injection attack against LLM-integrated applications,” arXiv preprint arXiv:2306.05499, 2023, doi: 10.48550/arXiv.2306.05499.
P. Manakul, A. Liusie, and M. J. F. Gales, “SelfCheckGPT: Zero-resource black-box hallucination detection for generative large language models,” in Proc. 2023 Conf. Empirical Methods in Natural Language Processing (EMNLP), Singapore, 2023, pp. 9004-9017, doi: 10.18653/v1/2023.emnlp-main.557.
OWASP Foundation, “OWASP Top 10 for Large Language Model Applications,” Version 1.1, 2023. [Online]. Available: https://owasp.org/www-project-top-10-for-large-language-model-applications/. [Accessed May 15, 2026].
T. Vu, D. Q. Nguyen, D. Q. Nguyen, M. Dras, and M. Johnson, “VnCoreNLP: A Vietnamese natural language processing toolkit,” in Proc. NAACL-HLT 2018: Demonstrations, New Orleans, LA, USA, 2018, pp. 56-60, doi: 10.18653/v1/N18-5012.
D. Q. Nguyen and A. T. Nguyen, “PhoBERT: Pre-trained language models for Vietnamese,” in Findings of the Association for Computational Linguistics: EMNLP 2020, 2020, pp. 1037-1042, doi: 10.18653/v1/2020.findings-emnlp.92.
S. Robertson and H. Zaragoza, “The probabilistic relevance framework: BM25 and beyond,” Foundations and Trends in Information Retrieval, vol. 3, no. 4, pp. 333-389, 2009, doi: 10.1561/1500000019.
J. Chen, S. Xiao, P. Zhang, K. Luo, D. Lian, and Z. Liu, “M3-Embedding: Multi-Linguality, Multi-Functionality, Multi-Granularity Text Embeddings Through Self-Knowledge Distillation,” in Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, 2024, pp. 2318-2335, doi: 10.18653/v1/2024.findings-acl.137.
Y. A. Malkov and D. A. Yashunin, “Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 42, no. 4, pp. 824-836, 2020, doi: 10.1109/TPAMI.2018.2889473.
W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica, “Efficient memory management for large language model serving with PagedAttention,” in Proc. 29th Symp. Operating Systems Principles (SOSP), Koblenz, Germany, 2023, pp. 611-626, doi: 10.1145/3600006.3613165.
