Natural Language Code Retrieval for 1C:Enterprise: An Open Benchmark and Efficient Bi-Encoder

2026-08-20 · Konstantin Chesnokov, Chingiz Mingazov arxiv

Natural language code retrieval is a rapidly evolving task in computer science. However, the 1C:Enterprise ecosystem combines Russian syntax with highly domain-specific terminology, for which open datasets and specialized models have been virtually non-existent. We present a comprehensive pipeline for 1C code retrieval: an open benchmark of 3,413 real-world, PII-scrubbed query-code pairs, a reproducible evaluation harness, and a specialized bi-encoder. To overcome scarce labeled data, we fine-tune on 784,057 synthetic triplets generated by google/gemma-4-26B-A4B-it from public code repositories, using Matryoshka Representation Learning (MRL) and a privacy-aware tokenizer. Because the benchmark subsets differ in size, we report balanced-subset macro, query-weighted micro, and forum-only results. Our model reaches 0.5992 balanced macro nDCG@10, 0.5044 micro, and 0.4617 on forum, versus 0.4932 macro for the baseline architecture and 0.5404 for google/embeddinggemma-300m. Removing every benchmark example flagged by the conservative exact/13-gram overlap audit leaves 0.6011 balanced macro (0.5010 micro), indicating that detected train-benchmark overlap does not explain the headline result. MRL truncation to 256 dimensions preserves 99.9% of retrieval quality while reducing dense-index storage and exact similarity arithmetic by a factor of three.

📄 PDF Abstract BibTeX arXiv:2608.19957

Code (0)

등록된 구현이 없습니다.

Tasks

Representation Learning

Evaluating the Efficacy of Open-Source LLMs in Enterprise-Specific RAG Systems: A Comparative Study of Performance and Scalability

2024-06-17 · Gautam B, Anupam Purwar

This paper presents an analysis of open-source large language models (LLMs) and their application in Retrieval-Augmented Generation (RAG) tasks, specific for enterprise-specific data sets scraped from their websites. Wit…

RAG Retrieval Retrieval-augmented Generation

CSR-RAG: An Efficient Retrieval System for Text-to-SQL on the Enterprise Scale

2026-01-10 · Rajpreet Singh, Novak Boškov, Lawrence Drabeck, Aditya Gudal 외 arxiv

Natural language to SQL translation (Text-to-SQL) is one of the long-standing problems that has recently benefited from advances in Large Language Models (LLMs). While most academic Text-to-SQL benchmarks request schema …

Table Retrieval

RAG Strategies for Natural Language-Based SQL Query and REST API Call Generation

2026-02-06 · Michael Marketsmüller, Simon Martin, Tim Schlippe arxiv

Enterprise software systems commonly expose business functionality through both relational databases and REST APIs. Accessing these interfaces requires specialized technical knowledge, as users must determine whether a r…

Code Generation

Reducing hallucination in structured outputs via Retrieval-Augmented Generation

2024-04-12 · Patrice Béchard, Orlando Marquez Ayala

A common and fundamental limitation of Generative AI (GenAI) is its propensity to hallucinate. While large language models (LLM) have taken the world by storm, without eliminating or at least reducing hallucinations, rea…

Hallucination RAG Retrieval Retrieval-augmented Generation

AgenticRAG: Agentic Retrieval for Enterprise Knowledge Bases

2026-05-07 · Susheel Suresh, Hazel Mak, Shangpo Chou, Fred Kroon 외 arxiv

We present AgenticRAG, a practical agentic harness for retrieval and analysis over enterprise knowledge bases. Standard RAG pipelines place significant burden of grounding on the search stack, constraining the language m…