paper-with-me

홈 › Papers

Character Jacobian: Modeling Chinese Character Meanings with Deep Learning Model

2022-10-01 · COLING 2022 10 · Yu-Hsiang Tseng, Shu-Kai Hsieh

Compounding, a prevalent word-formation process, presents an interesting challenge for computational models. Indeed, the relations between compounds and their constituents are often complicated. It is particularly so in Chinese morphology, where each character is almost simultaneously bound and free when treated as a morpheme. To model such word-formation process, we propose the Notch (NOnlinear Transformation of CHaracter embeddings) model and the character Jacobians. The Notch model first learns the non-linear relations between the constituents and words, and the character Jacobians further describes the character’s role in each word. In a series of experiments, we show that the Notch model predicts the embeddings of the real words from their constituents but helps account for the behavioral data of the pseudowords. Moreover, we also demonstrated that character Jacobians reflect the characters’ meanings. Taken together, the Notch model and character Jacobians may provide a new perspective on studying the word-formation process and morphology with modern deep learning.

📄 PDF Abstract BibTeX

Code (0)

등록된 구현이 없습니다.

Tasks

Deep Learning

Similar Papers 제목 키워드 기반

Character-level Chinese Backpack Language Models

2023-10-19 · Hao Sun, John Hewitt

The Backpack is a Transformer alternative shown to improve interpretability in English language modeling by decomposing predictions into a weighted sum of token sense components. However, Backpacks' reliance on token-def…

Language ModelingLanguage Modelling

The Dependence of Frequency Distributions on Multiple Meanings of Words, Codes and Signs

2017-09-28 · Xiao-Yong Yan, Petter Minnhagen

The dependence of the frequency distributions due to multiple meanings of words in a text is investigated by deleting letters. By coding the words with fewer letters the number of meanings per coded word increases. This …

Maximum Entropy, Word-Frequency, Chinese Characters, and Multiple Meanings

2014-02-09 · Xiao-Yong Yan, Petter Minnhagen

The word-frequency distribution of a text written by an author is well accounted for by a maximum entropy distribution, the RGF (random group formation)-prediction. The RGF-distribution is completely determined by the a …

Prediction

Robust Chinese Word Segmentation with Contextualized Word Representations

2019-01-17 · Yung-Sung Chuang

In recent years, after the neural-network-based method was proposed, the accuracy of the Chinese word segmentation task has made great progress. However, when dealing with out-of-vocabulary words, there is still a large …

Chinese Word SegmentationLanguage ModelingLanguage Modelling

ChineseBERT: Chinese Pretraining Enhanced by Glyph and Pinyin Information

2021-06-30 · ACL 2021 5 · Zijun Sun, Xiaoya Li, Xiaofei Sun, Yuxian Meng 외

Recent pretraining models in Chinese neglect two important aspects specific to the Chinese language: glyph and pinyin, which carry significant syntax and semantic information for language understanding. In this work, we …

Language ModelingLanguage ModellingMachine Reading ComprehensionNamed Entity Recognition+5