ELTEX: A Framework for Domain-Driven Synthetic Data Generation

2025-03-19 · Arina Razmyslovich, Kseniia Murasheva, Sofia Sedlova, Julien Capitaine, Eugene Dmitriev

We introduce Efficient LLM Token Extraction (ELTEX), a framework addressing the critical challenge of LLM domain specialization by systematically extracting and integrating domain indicators throughout synthetic data generation. Unlike approaches relying on implicit knowledge transfer, ELTEX explicitly leverages domain signals to maintain specialized knowledge integrity. In our cybersecurity case study, ELTEX-enhanced data enables a fine-tuned Gemma-2B model to achieve performance competitive with GPT-4o on blockchain cyberattack classification while reducing computational requirements. Our Google Sheets implementation makes ELTEX accessible to non-technical users. Our contributions include: (1) the ELTEX framework; (2) Google Sheets Add-on implementation; (3) empirical validation showing how ELTEX bridges performance gaps between small and large models; and (4) a synthetic dataset of 11,448 texts for blockchain cyberattack detection.

📄 PDF Abstract BibTeX arXiv:2503.15055

Code (1)

1712n/eltex 공식 구현

Tasks

Synthetic Data Generation Transfer Learning

Methods 이 논문이 사용한 방법론

Attention 설명 없음

Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.

Position-Wise Feed-Forward Layer 설명 없음

Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…

BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Softmax The Softmax output function transforms a previous layer's output into a vector of probabilities. It is commonly used for multiclass classification. Given an input vector $x$…

Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…

Multi-Head Attention 설명 없음

RelTextRank: An Open Source Framework for Building Relational Syntactic-Semantic Text Pair Representations

2017-07-01 · ACL 2017 7 · Kateryna Tymoshenko, Aless Moschitti, ro, Massimo Nicosia 외

Community Question Answering Natural Language Inference Question Answering Relational Reasoning

Everything to the Synthetic: Diffusion-driven Test-time Adaptation via Synthetic-Domain Alignment

2024-06-06 · CVPR 2025 1 · Jiayi Guo, Junhao Zhao, Chaoqun Du, Yulin Wang 외

Test-time adaptation (TTA) aims to improve the performance of source-domain pre-trained models on previously unseen, shifted target domains. Traditional TTA methods primarily adapt model weights based on target data stre…

Test-time Adaptation

Integrating Expert Knowledge with Domain Adaptation for Unsupervised Fault Diagnosis

2021-07-05 · Qin Wang, Cees Taal, Olga Fink

Data-driven fault diagnosis methods often require abundant labeled examples for each fault type. On the contrary, real-world data is often unlabeled and consists of mostly healthy observations and only few samples of fau…

Domain Adaptation Fault Diagnosis

A Data-Driven Probabilistic Framework for Cascading Urban Risk Analysis Using Bayesian Networks

2025-05-07 · Chunduru Rohith Kumar, PHD Surya Shanmuk, Prabhala Naga Srinivas, Sri Venkatesh Lankalapalli 외

The increasing complexity of cascading risks in urban systems necessitates robust, data-driven frameworks to model interdependencies across multiple domains. This study presents a foundational Bayesian network-based appr…

No more hard prompts: SoftSRV prompting for synthetic data generation

2024-10-21 · Giulia Desalvo, Jean-Fracois Kagy, Lazaros Karydas, Afshin Rostamizadeh 외

We present a novel soft prompt based framework, SoftSRV, that leverages a frozen pre-trained large language model (LLM) to generate targeted synthetic text sequences. Given a sample from the target distribution, our prop…

Language Modeling Language Modelling Large Language Model Math+1