Automatic Curation of Golf Highlights using Multimodal Excitement Features
The production of sports highlight packages summarizing a game's most exciting moments is an essential task for broadcast media. Yet, it requires labor-intensive video editing. We propose a novel approach for auto-curating sports highlights, and use it to create a real-world system for the editorial aid of golf highlight reels. Our method fuses information from the players' reactions (action recognition such as high-fives and fist pumps), spectators (crowd cheering), and commentator (tone of the voice and word analysis) to determine the most interesting moments of a game. We accurately identify the start and end frames of key shot highlights with additional metadata, such as the player's name and the hole number, allowing personalized content summarization and retrieval. In addition, we introduce new techniques for learning our classifiers with reduced manual training data annotation by exploiting the correlation of different modalities. Our work has been demonstrated at a major golf tournament, successfully extracting highlights from live video streams over four consecutive days.
Code (0)
등록된 구현이 없습니다.
Tasks
Action RecognitionRetrievalTemporal Action LocalizationVideo EditingSimilar Papers 제목 키워드 기반
Set Constraint Model and Automated Encoding into SAT: Application to the Social Golfer Problem
On the one hand, Constraint Satisfaction Problems allow one to declaratively model problems. On the other hand, propositional satisfiability problem (SAT) solvers can handle huge SAT instances. We thus present a techniqu…
GolfDB: A Video Database for Golf Swing Sequencing
The golf swing is a complex movement requiring considerable full-body coordination to execute proficiently. As such, it is the subject of frequent scrutiny and extensive biomechanical analyses. In this paper, we introduc…
An Evaluation of GPT-4V and Gemini in Online VQA
While there is much excitement about the potential of large multimodal models (LMM), a comprehensive evaluation is critical to establish their true capabilities and limitations. In support of this aim, we evaluate two st…
Question AnsweringVisual Question AnsweringVisual Question Answering (VQA)RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model
Minigolf is an exemplary real-world game for examining embodied intelligence, requiring challenging spatial and kinodynamic understanding to putt the ball. Additionally, reflective reasoning is required if the feasibilit…
Language ModelingLanguage ModellingCountries across the world use more land for golf courses than wind or solar energy
Land use is a critical factor in the siting of renewable energy facilities and is often scrutinized due to perceived conflicts with other land demands. Meanwhile, substantial areas are devoted to activities such as golf,…