paper-with-me

Papers

Rank-Minimizing and Structured Model Inference

2023-02-19 · Pawan Goyal, Benjamin Peherstorfer, Peter Benner

While extracting information from data with machine learning plays an increasingly important role, physical laws and other first principles continue to provide critical insights about systems and processes of interest in science and engineering. This work introduces a method that infers models from data with physical insights encoded in the form of structure and that minimizes the model order so that the training data are fitted well while redundant degrees of freedom without conditions and sufficient data to fix them are automatically eliminated. The models are formulated via solution matrices of specific instances of generalized Sylvester equations that enforce interpolation of the training data and relate the model order to the rank of the solution matrices. The proposed method numerically solves the Sylvester equations for minimal-rank solutions and so obtains models of low order. Numerical experiments demonstrate that the combination of structure preservation and rank minimization leads to accurate models with orders of magnitude fewer degrees of freedom than models of comparable prediction quality that are learned with structure preservation alone.

📄 PDF Abstract BibTeX arXiv:2302.09521

Code (0)

등록된 구현이 없습니다.

Tasks

model

Similar Papers 제목 키워드 기반

Gumbel Reranking: Differentiable End-to-End Reranker Optimization

2025-02-16 · Siyuan Huang, Zhiyuan Ma, Jintao Du, Changhua Meng 외

RAG systems rely on rerankers to identify relevant documents. However, fine-tuning these models remains challenging due to the scarcity of annotated query-document pairs. Existing distillation-based approaches suffer fro…

RAGReranking

Tighter Bounds for Structured Estimation

2008-12-01 · NeurIPS 2008 12 · Olivier Chapelle, Chuong B. Do, Choon H. Teo, Quoc V. Le 외

Large-margin structured estimation methods work by minimizing a convex upper bound of loss functions. While they allow for efficient optimization algorithms, these convex formulations are not tight and sacrifice the abil…

Binary ClassificationGeneral ClassificationStructured Prediction

Train Large, Deploy Compact: Structured Compression for Compact Low-Rank Adaptation

2025-09-30 · Xin Yu, Cong Xie, Ziyu Zhao, Tiantian Fan 외 arxiv

Low-rank adaptation (LoRA) has become a widely used paradigm for parameter-efficient fine-tuning of large language models, yet its representational capacity often lags behind full fine-tuning. Within the context of LoRA,…

parameter-efficient fine-tuningNatural Language UnderstandingMathematical ReasoningCode Generation

A Study of Latent Structured Prediction Approaches to Passage Reranking

2019-06-01 · NAACL 2019 6 · Iryna Haponchyk, Aless Moschitti, ro

The structured output framework provides a helpful tool for learning to rank problems. In this paper, we propose a structured output approach which regards rankings as latent variables. Our approach addresses the complex…

Learning-To-RankPassage RerankingRerankingStructured Prediction

Dynamic Programming in Rank Space: Scaling Structured Inference with Low-Rank HMMs and PCFGs

2022-05-01 · NAACL 2022 7 · Songlin Yang, Wei Liu, Kewei Tu

Hidden Markov Models (HMMs) and Probabilistic Context-Free Grammars (PCFGs) are widely used structured models, both of which can be represented as factor graph grammars (FGGs), a powerful formalism capable of describing …

Constituency Grammar InductionLanguage ModelingLanguage Modelling