Task Graph Maximum Likelihood Estimation for Procedural Activity Understanding in Egocentric Videos
We introduce a gradient-based approach for learning task graphs from procedural activities, improving over hand-crafted methods. Our method directly optimizes edge weights via maximum likelihood, enabling integration into neural architectures. We validate our approach on CaptainCook4D, EgoPER, and EgoProceL, achieving +14.5%, +10.2%, and +13.6% F1-score improvements. Our feature-based approach for predicting task graphs from textual/video embeddings demonstrates emerging video understanding abilities. We also achieved top performance on the procedure understanding benchmark on Ego-Exo4D and significantly improved online mistake detection (+19.8% on Assembly101-O, +6.4% on EPIC-Tent-O). Code: https://github.com/fpv-iplab/Differentiable-Task-Graph-Learning.
Code (1)
Tasks
Graph LearningMistake DetectionOnline Mistake DetectionTask Graph LearningVideo UnderstandingSimilar Papers 제목 키워드 기반
Composite Likelihood Estimation for Restricted Boltzmann machines
Learning the parameters of graphical models using the maximum likelihood estimation is generally hard which requires an approximation. Maximum composite likelihood estimations are statistical approximations of the maximu…
Differentiable Task Graph Learning: Procedural Activity Representation and Online Mistake Detection from Egocentric Videos
Procedural activities are sequences of key-steps aimed at achieving specific goals. They are crucial to build intelligent agents able to assist users effectively. In this context, task graphs have emerged as a human-unde…
Mistake DetectionOnline Mistake DetectionTask Graph LearningVideo UnderstandingMaximum likelihood thresholds of Gaussian graphical models and graphical lasso
Associated to each graph G is a Gaussian graphical model. Such models are often used in high-dimensional settings, i.e. where there are relatively few data points compared to the number of variables. The maximum likeliho…
A local approach to estimation in discrete loglinear models
We consider two connected aspects of maximum likelihood estimation of the parameter for high-dimensional discrete graphical models: the existence of the maximum likelihood estimate (mle) and its computation. When the d…
Multimodal Image Synthesis with Conditional Implicit Maximum Likelihood Estimation
Many tasks in computer vision and graphics fall within the framework of conditional image synthesis. In recent years, generative adversarial nets (GANs) have delivered impressive advances in quality of synthesized images…
Image GenerationImage Super-ResolutionSuper-Resolution