paper-with-me

Papers

Multi-Objective Instruction-Aware Representation Learning in Procedural Content Generation RL

2025-08-08 · Sung-Hyun Kim, Geum-Hwan Hwang, In-Chang Baek, Seo-Young Lee, Kyung-Joong Kim arxiv

Recent advancements in generative modeling emphasize the importance of natural language as a highly expressive and accessible modality for controlling content generation. However, existing instructed reinforcement learning for procedural content generation (IPCGRL) method often struggle to leverage the expressive richness of textual input, especially under complex, multi-objective instructions, leading to limited controllability. To address this problem, we propose \textit{MIPCGRL}, a multi-objective representation learning method for instructed content generators, which incorporates sentence embeddings as conditions. MIPCGRL effectively trains a multi-objective embedding space by incorporating multi-label classification and multi-head regression networks. Experimental results show that the proposed method achieves up to a 13.8\% improvement in controllability with multi-objective instructions. The ability to process complex instructions enables more expressive and flexible content generation.

📄 PDF Abstract BibTeX arXiv:2508.09193

Code (0)

등록된 구현이 없습니다.

Tasks

Multi-Label ClassificationRepresentation LearningReinforcement Learning

Similar Papers 제목 키워드 기반

Procedure-Aware Pretraining for Instructional Video Understanding

2023-03-31 · CVPR 2023 1 · Honglu Zhou, Roberto Martín-Martín, Mubbasir Kapadia, Silvio Savarese 외

Our goal is to learn a video representation that is useful for downstream procedure understanding tasks in instructional videos. Due to the small amount of available annotations, a key challenge in procedure understandin…

Video Understanding

Constructing Evaluation Datasets for Procedural Reasoning: Balancing Naturalness, Grounding, and Multi-Hop Coverage

2026-06-11 · Sarah Elshabrawy, Rahul K. Dass, Ashok K. Goel arxiv

Evaluating procedural reasoning in AI-supported learning systems requires question-answer datasets that are both learner-like and grounded in the instructional knowledge the system is expected to use. We study how TMK-ba…

Question Generation

Less is More: Label-Guided Summarization of Procedural and Instructional Videos

2026-01-18 · Shreya Rajpal, Michal Golovanevsky, Carsten Eickhoff arxiv

Video summarization helps turn long videos into clear, concise representations that are easier to review, document, and analyze, especially in high-stakes domains like surgical training. Prior work has progressed from us…

Video Summarization

Automatically Extracting Procedural Knowledge from Instructional Texts using Natural Language Processing

2012-05-01 · LREC 2012 5 · Ziqi Zhang, Philip Webster, Victoria Uren, Andrea Varga 외

Procedural knowledge is the knowledge required to perform certain tasks, and forms an important part of expertise. A major source of procedural knowledge is natural language instructions. While these readable instruction…

Question Answering

CrochetBench: Can Vision-Language Models Move from Describing to Doing in Crochet Domain?

2025-11-12 · Peiyu Li, Xiaobao Huang, Ting Hua, Nitesh V. Chawla arxiv

While multimodal large language models can describe visual content, their ability to generate executable procedures remains underexplored. CrochetBench presented in this paper evaluates this shift from describing to doin…