Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language
We present a new pre-trained language model (PLM) for Rabbinic Hebrew, termed Berel (BERT Embeddings for Rabbinic-Encoded Language). Whilst other PLMs exist for processing Hebrew texts (e.g., HeBERT, AlephBert), they are all trained on modern Hebrew texts, which diverges substantially from Rabbinic Hebrew in terms of its lexicographical, morphological, syntactic and orthographic norms. We demonstrate the superiority of Berel on Rabbinic texts via a challenge set of Hebrew homographs. We release the new model and homograph challenge set for unrestricted use.
Code (0)
등록된 구현이 없습니다.
Tasks
Language ModelingLanguage ModellingSimilar Papers 제목 키워드 기반
Probing with Noise: Unpicking the Warp and Weft of Embeddings
Improving our understanding of how information is encoded in vector space can yield valuable interpretability insights. Alongside vector dimensions, we argue that it is possible for the vector norm to also carry linguist…
SentenceStyle Classification of Rabbinic Literature for Detection of Lost Midrash Tanhuma Material
Midrash collections are complex rabbinic works that consist of text in multiple languages, which evolved through long processes of unstable oral and written transmission. Determining the origin of a given passage in such…
Contextual and Non-Contextual Word Embeddings: an in-depth Linguistic Investigation
In this paper we present a comparison between the linguistic knowledge encoded in the internal representations of a contextual Language Model (BERT) and a contextual-independent one (Word2vec). We use a wide set of probi…
Language ModelingLanguage ModellingSentenceWord EmbeddingsIntroducing DictaLM -- A Large Generative Language Model for Modern Hebrew
We present DictaLM, a large-scale language model tailored for Modern Hebrew. Boasting 7B parameters, this model is predominantly trained on Hebrew-centric data. As a commitment to promoting research and development in th…
Language ModelingLanguage ModellingSentiment AnalysisLex-BERT: Enhancing BERT based NER with lexicons
In this work, we represent Lex-BERT, which incorporates the lexicon information into Chinese BERT for named entity recognition (NER) tasks in a natural manner. Instead of using word embeddings and a newly designed transf…
named-entity-recognitionNamed Entity RecognitionNamed Entity Recognition (NER)NER+2