ESGBERT: Language Model to Help with Classification Tasks Related to Companies Environmental, Social, and Governance Practices
Environmental, Social, and Governance (ESG) are non-financial factors that are garnering attention from investors as they increasingly look to apply these as part of their analysis to identify material risks and growth opportunities. Some of this attention is also driven by clients who, now more aware than ever, are demanding for their money to be managed and invested responsibly. As the interest in ESG grows, so does the need for investors to have access to consumable ESG information. Since most of it is in text form in reports, disclosures, press releases, and 10-Q filings, we see a need for sophisticated NLP techniques for classification tasks for ESG text. We hypothesize that an ESG domain-specific pre-trained model will help with such and study building of the same in this paper. We explored doing this by fine-tuning BERTs pre-trained weights using ESG specific text and then further fine-tuning the model for a classification task. We were able to achieve accuracy better than the original BERT and baseline models in environment-specific classification tasks.
Code (0)
등록된 구현이 없습니다.
Tasks
ClassificationLanguage ModelingLanguage ModellingMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
A Multi-task Model for Multilingual Trigger Detection and Classification
In this paper we present a deep multi-task learning framework for multilingual event and argument trigger detection and classification. In our current work, we identify detection and classification of both event and argu…
ClassificationMulti-Task LearningRecurrent Neural Network for Text Classification with Multi-Task Learning
Neural network based methods have obtained great progress on a variety of natural language processing tasks. However, in most previous works, the models are learned based on single-task supervised objectives, which often…
ClassificationEmotion Recognition in ConversationGeneral ClassificationMulti-Task Learning+2Cross-lingual Transfer for Speech Processing using Acoustic Language Similarity
Speech processing systems currently do not support the vast majority of languages, in part due to the lack of data in low-resource languages. Cross-lingual transfer offers a compelling way to help bridge this digital div…
Cross-Lingual Transferspeech-recognitionSpeech RecognitionSpeech SynthesisData Augmentation for Intent Classification with Off-the-shelf Large Language Models
Data augmentation is a widely employed technique to alleviate the problem of data scarcity. In this work, we propose a prompting-based approach to generate labelled training data for intent classification with off-the-sh…
Data Augmentationintent-classificationIntent ClassificationImproving Downstream Task Performance by Treating Numbers as Entities
Numbers are essential components of text, like any other word tokens, from which natural language processing (NLP) models are built and deployed. Though numbers are typically not accounted for distinctly in most NLP task…
ClassificationQuestion Answering