Multi-Granularity Contrasting for Cross-Lingual Pre-Training
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
VECO 2.0: Cross-lingual Language Model Pre-training with Multi-granularity Contrastive Learning
Recent studies have demonstrated the potential of cross-lingual transferability by training a unified Transformer encoder for multiple languages. In addition to involving the masked language model objective, existing cro…
Contrastive LearningLanguage ModelingLanguage ModellingSentenceM2G-Eval: Enhancing and Evaluating Multi-granularity Multilingual Code Generation
The rapid advancement of code large language models (LLMs) has sparked significant research interest in systematically evaluating their code generation capabilities, yet existing benchmarks predominantly assess models at…
Code GenerationInfoXLM: An Information-Theoretic Framework for Cross-Lingual Language Model Pre-Training
In this work, we present an information-theoretic framework that formulates cross-lingual language model pre-training as maximizing mutual information between multilingual-multi-granularity texts. The unified view helps …
Contrastive LearningCross-Lingual TransferLanguage ModelingLanguage Modelling+1HTG-GCL: Leveraging Hierarchical Topological Granularity from Cellular Complexes for Graph Contrastive Learning
Graph contrastive learning (GCL) aims to learn discriminative semantic invariance by contrasting different views of the same graph that share critical topological patterns. However, existing GCL approaches with structura…
Contrastive LearningZero-Shot Cross-Lingual Document-Level Event Causality Identification with Heterogeneous Graph Contrastive Transfer Learning
Event Causality Identification (ECI) refers to the detection of causal relations between events in texts. However, most existing studies focus on sentence-level ECI with high-resource languages, leaving more challenging …
Event Causality IdentificationFew-Shot LearningSentenceTransfer Learning