Divide and Recombine for Large and Complex Data: Model Likelihood Functions using MCMC

2018-01-15 · Qi Liu, Anindya Bhadra, William S. Cleveland

In Divide & Recombine (D&R), big data are divided into subsets, each analytic method is applied to subsets, and the outputs are recombined. This enables deep analysis and practical computational performance. An innovate D\&R procedure is proposed to compute likelihood functions of data-model (DM) parameters for big data. The likelihood-model (LM) is a parametric probability density function of the DM parameters. The density parameters are estimated by fitting the density to MCMC draws from each subset DM likelihood function, and then the fitted densities are recombined. The procedure is illustrated using normal and skew-normal LMs for the logistic regression DM.

📄 PDF Abstract BibTeX arXiv:1801.05007

Code (0)

등록된 구현이 없습니다.

Tasks

regression

Methods 이 논문이 사용한 방법론

Logistic Regression Logistic Regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression,…

Hey, wait a minute: on at-issue sensitivity in Language Models

2025-10-14 · Sanghee J. Kim, Kanishka Misra arxiv

Evaluating the naturalness of dialogue in language models (LMs) is not trivial: notions of 'naturalness' vary, and scalable quantitative metrics remain limited. This study leverages the linguistic notion of 'at-issueness…

SwISS: A Scalable Markov chain Monte Carlo Divide-and-Conquer Strategy

2022-08-08 · Callum Vyner, Christopher Nemeth, Chris Sherlock

Divide-and-conquer strategies for Monte Carlo algorithms are an increasingly popular approach to making Bayesian inference scalable to large data sets. In its simplest form, the data are partitioned across multiple compu…

Bayesian Inference

Realization of long sentences using chunking

2017-09-01 · WS 2017 9 · Ewa Muszy{\'n}ska, Ann Copestake

We propose sentence chunking as a way to reduce the time and memory costs of realization of long sentences. During chunking we divide the semantic representation of a sentence into smaller components which can be process…

Chunking Sentence Text Generation

The DipInfoUniTo Realizer at SRST'19: Learning to Rank and Deep Morphology Prediction for Multilingual Surface Realization

2019-11-01 · WS 2019 11 · Aless Mazzei, ro, Valerio Basile

We describe the system presented at the SR{'}19 shared task by the DipInfoUnito team. Our approach is based on supervised machine learning. In particular, we divide the SR task into two independent subtasks, namely word …

BIG-bench Machine Learning Learning-To-Rank Prediction

RankEvolve: Automating the Discovery of Retrieval Algorithms via LLM-Driven Evolution

2026-02-18 · Jinming Nian, Fangchen Li, Dae Hoon Park, Yi Fang arxiv

Retrieval algorithms like BM25 and query likelihood with Dirichlet smoothing remain strong and efficient first-stage rankers, yet improvements have mostly relied on parameter tuning and human intuition. We investigate wh…