paper-with-me

홈 › Papers

Curriculum Recommendations Using Transformer Base Model with InfoNCE Loss And Language Switching Method

2024-01-18 · Xiaonan Xu, Bin Yuan, Yongyao Mo, Tianbo Song, Shulin Li

The Curriculum Recommendations paradigm is dedicated to fostering learning equality within the ever-evolving realms of educational technology and curriculum development. In acknowledging the inherent obstacles posed by existing methodologies, such as content conflicts and disruptions from language translation, this paradigm aims to confront and overcome these challenges. Notably, it addresses content conflicts and disruptions introduced by language translation, hindrances that can impede the creation of an all-encompassing and personalized learning experience. The paradigm's objective is to cultivate an educational environment that not only embraces diversity but also customizes learning experiences to suit the distinct needs of each learner. To overcome these challenges, our approach builds upon notable contributions in curriculum development and personalized learning, introducing three key innovations. These include the integration of Transformer Base Model to enhance computational efficiency, the implementation of InfoNCE Loss for accurate content-topic matching, and the adoption of a language switching strategy to alleviate translation-related ambiguities. Together, these innovations aim to collectively tackle inherent challenges and contribute to forging a more equitable and effective learning journey for a diverse range of learners. Competitive cross-validation scores underscore the efficacy of sentence-transformers/LaBSE, achieving 0.66314, showcasing our methodology's effectiveness in diverse linguistic nuances for content alignment prediction. Index Terms-Curriculum Recommendation, Transformer model with InfoNCE Loss, Language Switching.

📄 PDF Abstract BibTeX arXiv:2401.09699

Code (0)

등록된 구현이 없습니다.

Tasks

Computational EfficiencySentenceTranslation

Methods 이 논문이 사용한 방법론

Multi-Head Attention 설명 없음
Attention 설명 없음
Label Smoothing Label Smoothing is a regularization technique that introduces noise for the labels. This accounts for the fact that datasets may have mistakes in them, so maximizing the…
Absolute Position Encodings Absolute Position Encodings are a type of position embeddings for [Transformer-based models] where positional encodings are…
Layer Normalization Unlike batch normalization, Layer Normalization directly estimates the normalization statistics from the summed inputs…
Dropout Dropout is a regularization technique for neural networks that drops a unit (along with connections) at training time with a specified probability $p$ (a common value is…
Linear Layer A Linear Layer is a projection $\mathbf{XW + b}$.
BPE Byte Pair Encoding, or BPE, is a subword segmentation algorithm that encodes rare and unknown words as sequences of subword units. The intuition is that various word…

Similar Papers 제목 키워드 기반

Empowering Collaborative Filtering with Principled Adversarial Contrastive Loss

2023-10-28 · NeurIPS 2023 11 · An Zhang, Leheng Sheng, Zhibo Cai, Xiang Wang 외

Contrastive Learning (CL) has achieved impressive performance in self-supervised learning tasks, showing superior generalization ability. Inspired by the success, adopting CL into collaborative filtering (CF) is prevaili…

Collaborative FilteringContrastive LearningData AugmentationRecommendation Systems+1

Interpretable Convolutional SyncNet

2024-09-02 · Sungjoon Park, Jaesub Yun, Donggeon Lee, Minsik Park

Because videos in the wild can be out of sync for various reasons, a sync-net is used to bring the video back into sync for tasks that require synchronized videos. Previous state-of-the-art (SOTA) sync-nets use InfoNCE l…

Temperature-Free Loss Function for Contrastive Learning

2025-01-29 · Bum Jun Kim, Sang Woo Kim

As one of the most promising methods in self-supervised learning, contrastive learning has achieved a series of breakthroughs across numerous fields. A predominant approach to implementing contrastive learning is applyin…

Contrastive LearningSelf-Supervised Learningvalid

Rethinking Negative Pairs in Code Search

2023-10-12 · Haochen Li, Xin Zhou, Luu Anh Tuan, Chunyan Miao

Recently, contrastive learning has become a key component in fine-tuning code search models for software development efficiency and effectiveness. It pulls together positive code snippets while pushing negative samples a…

Code SearchContrastive LearningMutual Information EstimationRepresentation Learning

Revisiting Recommendation Loss Functions through Contrastive Learning (Technical Report)

2023-12-13 · Dong Li, Ruoming Jin, Bin Ren

Inspired by the success of contrastive learning, we systematically examine recommendation losses, including listwise (softmax), pairwise (BPR), and pointwise (MSE and CCL) losses. In this endeavor, we introduce InfoNCE+,…

Contrastive Learning