PatentBERT: Patent Classification with Fine-Tuning a pre-trained BERT Model
In this work we focus on fine-tuning a pre-trained BERT model and applying it to patent classification. When applied to large datasets of over two millions patents, our approach outperforms the state of the art by an approach using CNN with word embeddings. In addition, we focus on patent claims without other parts in patent documents. Our contributions include: (1) a new state-of-the-art method based on pre-trained BERT model and fine-tuning for patent classification, (2) a large dataset USPTO-3M at the CPC subclass level with SQL statements that can be used by future researchers, (3) showing that patent claims alone are sufficient for classification task, in contrast to conventional wisdom.
Code (1)
Tasks
ClassificationGeneral ClassificationMulti-Label Text ClassificationPatent classificationMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Patent Language Model Pretraining with ModernBERT
Transformer-based language models such as BERT have become foundational in NLP, yet their performance degrades in specialized domains like patents, which contain long, technical, and legally structured text. Prior approa…
Measuring Patent Claim Generation by Span Relevancy
Our goal of patent claim generation is to realize "augmented inventing" for inventors by leveraging latest Deep Learning techniques. We envision the possibility of building an "auto-complete" function for inventors to co…
Language ModellingNatural Language InferenceText GenerationUniversal Language Model Fine-tuning for Patent Classification
This paper describes the methods used for the 2018 ALTA Shared Task. The task this year was to automatically classify Australian patents into their main International Patent Classification section. Our final submission u…
ClassificationGeneral ClassificationLanguage ModelingLanguage Modelling+3Improving Patent Mining and Relevance Classification using Transformers
Patent analysis and mining are time-consuming and costly processes for companies, but nevertheless essential if they are willing to remain competitive. To face the overload induced by numerous patents, the idea is to aut…
ClassificationGeneral ClassificationPatent classificationPatent Claim Generation by Fine-Tuning OpenAI GPT-2
In this work, we focus on fine-tuning an OpenAI GPT-2 pre-trained model for generating patent claims. GPT-2 has demonstrated impressive efficacy of pre-trained language models on various tasks, particularly coherent text…
Text Generation