Looks Right, Works Right: A Project-Level Benchmark for Multi-Screen Mobile App Generation
Recent multimodal large language models can convert visual designs directly into executable code, but real mobile products require multiple screenshots to become a buildable codebase with shared components and working navigation. This project-level setting exposes three limits of existing design-to-code benchmarks: they focus on single-page generation rather than complete codebases, cannot evaluate cross-page navigation, and do not measure project-wide maintainability. We introduce MobileForge, the first benchmark for project-level multi-screen mobile app generation, comprising real mobile apps, human-reviewed screens, structured page-relationship annotations, and navigation test specifications. MobileForge supports five-axis evaluation of build, navigation, visual fidelity, code maintainability, and efficiency. We also propose state-isolated navigation testing to avoid cascading failures in navigation evaluation and an anchor-referenced list-wise visual evaluation protocol to improve visual-judge reliability. Across end-to-end runs on six frontier multimodal LLMs, current models can build mobile-app projects that compile and reach the correct pages, but interactive navigation remains unreliable and visual fidelity and maintainability still lag. The benchmark and supporting materials are available at https://github.com/anoa12159-hue/mobileforge_eval.
Code (0)
등록된 구현이 없습니다.
Similar Papers 제목 키워드 기반
Disguised Copyright Infringement of Latent Diffusion Models
Copyright infringement may occur when a generative model produces samples substantially similar to some copyrighted data that it had access to during the training phase. The notion of access usually refers to including c…
A Priori Generalizability Estimate for a CNN
We formulate truncated singular value decompositions of entire convolutional neural networks. We demonstrate the computed left and right singular vectors are useful in identifying which images the convolutional neural ne…
Diagnosticimage-classificationImage ClassificationImage Segmentation+1SCOPE: Intrinsic Semantic Space Control for Mitigating Copyright Infringement in LLMs
Large language models sometimes inadvertently reproduce passages that are copyrighted, exposing downstream applications to legal risk. Most existing studies for inference-time defences focus on surface-level token matchi…
A Plug-and-Play Defensive Perturbation for Copyright Protection of DNN-based Applications
Wide deployment of deep neural networks (DNNs) based applications (e.g., style transfer, cartoonish), stimulating the requirement of copyright protection of such application's production. Although some traditional visibl…
DecoderStyle TransferDistilling the Knowledge of BERT for Sequence-to-Sequence ASR
Attention-based sequence-to-sequence (seq2seq) models have achieved promising results in automatic speech recognition (ASR). However, as these models decode in a left-to-right way, they do not have access to context on t…
Automatic Speech RecognitionAutomatic Speech Recognition (ASR)Knowledge DistillationLanguage Modeling+3