BERT-PIN: A BERT-based Framework for Recovering Missing Data Segments in Time-series Load Profiles
Inspired by the success of the Transformer model in natural language processing and computer vision, this paper introduces BERT-PIN, a Bidirectional Encoder Representations from Transformers (BERT) powered Profile Inpainting Network. BERT-PIN recovers multiple missing data segments (MDSs) using load and temperature time-series profiles as inputs. To adopt a standard Transformer model structure for profile inpainting, we segment the load and temperature profiles into line segments, treating each segment as a word and the entire profile as a sentence. We incorporate a top candidates selection process in BERT-PIN, enabling it to produce a sequence of probability distributions, based on which users can generate multiple plausible imputed data sets, each reflecting different confidence levels. We develop and evaluate BERT-PIN using real-world dataset for two applications: multiple MDSs recovery and demand response baseline estimation. Simulation results show that BERT-PIN outperforms the existing methods in accuracy while is capable of restoring multiple MDSs within a longer window. BERT-PIN, served as a pre-trained model, can be fine-tuned for conducting many downstream tasks, such as classification and super resolution.
Code (0)
등록된 구현이 없습니다.
Tasks
SentenceSuper-ResolutionTime SeriesMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Retrieving Skills from Job Descriptions: A Language Model Based Extreme Multi-label Classification Framework
We introduce a deep learning model to learn the set of enumerated job skills associated with a job description. In our analysis of a large-scale government job portal mycareersfuture.sg, we observe that as much as 65{\%}…
Extreme Multi-Label ClassificationLanguage ModelingLanguage ModellingMulti-Label Classification+1GiBERT: Introducing Linguistic Knowledge into BERT through a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks. However, BERT is only trained to predict missing words - either behind masks or in the next sen…
Semantic SimilaritySemantic Textual SimilaritySentenceUnsupervised Pre-training+1GenoBERT: A Language Model for Accurate Genotype Imputation
Genotype imputation enables dense variant coverage for genome-wide association and risk-prediction studies, yet conventional reference-panel methods remain limited by ancestry bias and reduced rare-variant accuracy. We p…
GiBERT: Enhancing BERT with Linguistic Information using a Lightweight Gated Injection Method
Large pre-trained language models such as BERT have been the driving force behind recent improvements across many NLP tasks. However, BERT is only trained to predict missing words – either through masking or next sentenc…
Semantic SimilaritySemantic Textual SimilaritySentenceUnsupervised Pre-training+1SeedBERT: Recovering Annotator Rating Distributions from an Aggregated Label
Many machine learning tasks -- particularly those in affective computing -- are inherently subjective. When asked to classify facial expressions or to rate an individual's attractiveness, humans may disagree with one ano…