TimeLens: On-Device Artifact Recognition with Retrieval-Augmented Question Answering for the Grand Egyptian Museum

2026-06-11 · Rawan Hesham, Ali Ashraf, Amr Ahmed, Malak Alaa, Omar Ahmed, Omar Wagih arxiv

TimeLens is an AI-powered bilingual mobile guide for the Grand Egyptian Museum (GEM). Pointing a phone at an exhibit, a visitor sees the artifact recognized in real time and can ask follow-up questions answered in English or Arabic. The work addresses three problems specific to in-gallery deployment: fine-grained visual similarity among 51 catalogued artifacts (many near-identical Ramesside statues), the gap between curated training data and handheld camera conditions, and the risk of an AI guide stating unsupported historical facts. Two engineering contributions are reported. First, an on-device artifact detector was developed through a data-quality-driven iteration study -- from foundation-model auto-annotation (YOLO-World), through spatial label-cleaning rules, to a fully hand-annotated dataset -- isolating label quality as the decisive factor: the final YOLOv8n model resolves every previously failing class while remaining a 5.97 MB TensorFlow Lite asset that runs in real time on a mid-range phone (mAP@0.5 = 0.995, mAP@0.5:0.95 = 0.924). Second, a bilingual Retrieval-Augmented Generation (RAG) guide, grounded in a 108-record ChromaDB knowledge base, was benchmarked across seven candidate language models, with Gemma 4 E2B (Q4 K M) selected; ten targeted optimizations reduce end-to-end latency from over 30 s to approximately 10 s. Both subsystems are integrated in a production Flutter application with bilingual interface, museum location gating, and text-to-speech support.

📄 PDF Abstract BibTeX arXiv:2606.13267

Code (0)

등록된 구현이 없습니다.

Tasks

Question Answering

TimeLens: Rethinking Video Temporal Grounding with Multimodal LLMs

2025-12-16 · Jun Zhang, Teng Wang, Yuying Ge, Yixiao Ge 외 arxiv

This paper does not introduce a novel method but instead establishes a straightforward, incremental, yet essential baseline for video temporal grounding (VTG), a core capability in video understanding. While multimodal l…

Reinforcement Learning

TimeLens2: Generalist Video Temporal Grounding with Multimodal LLMs

2026-07-19 · Yuhan Zhu, Changlian Ma, Xiangyu Zeng, Xinhao Li 외 hf

Video multimodal large language models (MLLMs) can describe what happens in a video, but rarely identify when the supporting evidence occurs. We study generalist video temporal grounding, in which one model predicts a va…

Retrieval-Augmented Speech Recognition Approach for Domain Challenges

2025-02-21 · Peng Shen, Xugang Lu, Hisashi Kawai

Speech recognition systems often face challenges due to domain mismatch, particularly in real-world applications where domain-specific data is unavailable because of data accessibility and confidentiality constraints. In…

Decoder RAG Retrieval Retrieval-augmented Generation+2

Provenance Analysis of Archaeological Artifacts via Multimodal RAG Systems

2025-09-25 · Tuo Zhang, Yuechun Sun, Ruiliang Liu arxiv

In this work, we present a retrieval-augmented generation (RAG)-based system for provenance analysis of archaeological artifacts, designed to support expert reasoning by integrating multimodal retrieval and large vision-…

Semantic Retrieval

Multi-Agent Transactive Memory

2026-06-18 · To Eun Kim, Xuhong He, Dishank Jain, Ambuj Agrawal 외 arxiv

The decentralized deployment of LLM agents with diverse capabilities across diverse tasks motivates infrastructure for knowledge sharing across heterogeneous agent populations. Just as search engines index human-generate…