VoxML: A Visualization Modeling Language

2016-10-05 · LREC 2016 5 · James Pustejovsky, Nikhil Krishnaswamy

We present the specification for a modeling language, VoxML, which encodes semantic knowledge of real-world objects represented as three-dimensional models, and of events and attributes related to and enacted over these objects. VoxML is intended to overcome the limitations of existing 3D visual markup languages by allowing for the encoding of a broad range of semantic knowledge that can be exploited by a variety of systems and platforms, leading to multimodal simulations of real-world scenarios using conceptual objects that represent their semantic values.

📄 PDF Abstract BibTeX arXiv:1610.01508

Code (0)

등록된 구현이 없습니다.

An Abstract Specification of VoxML as an Annotation Language

2023-05-22 · Kiyong Lee, Nikhil Krishnaswamy, James Pustejovsky

VoxML is a modeling language used to map natural language expressions into real-time visualizations using commonsense semantic knowledge of objects and events. Its utility has been demonstrated in embodied simulation env…

Human Agent Collaboration Human-Object Interaction Detection Object

Building Multimodal Simulations for Natural Language

2017-04-01 · EACL 2017 4 · James Pustejovsky, Nikhil Krishnaswamy

In this tutorial, we introduce a computational framework and modeling language (VoxML) for composing multimodal simulations of natural language expressions within a 3D simulation environment (VoxSim). We demonstrate how …

Formal Logic Referring Expression Referring expression generation Scene Generation+1

ECAT: Event Capture Annotation Tool

2016-10-05 · Tuan Do, Nikhil Krishnaswamy, James Pustejovsky

This paper introduces the Event Capture Annotation Tool (ECAT), a user-friendly, open-source interface tool for annotating events and their participants in video, capable of extracting the 3D positions and orientations o…

Attribute

The VoxWorld Platform for Multimodal Embodied Agents

2022-06-01 · LREC 2022 6 · Nikhil Krishnaswamy, William Pickard, Brittany Cates, Nathaniel Blanchard 외

We present a five-year retrospective on the development of the VoxWorld platform, first introduced as a multimodal platform for modeling motion language, that has evolved into a platform for rapidly building and deployin…

multimodal interaction

The Development of Multimodal Lexical Resources

2016-12-01 · WS 2016 12 · James Pustejovsky, Tuan Do, Gitit Kehat, Nikhil Krishnaswamy

Human communication is a multimodal activity, involving not only speech and written expressions, but intonation, images, gestures, visual clues, and the interpretation of actions through perception. In this paper, we des…

Question Answering Visual Question Answering (VQA)