A Multi-objective Optimization Benchmark Test Suite for Real-time Semantic Segmentation
As one of the emerging challenges in Automated Machine Learning, the Hardware-aware Neural Architecture Search (HW-NAS) tasks can be treated as black-box multi-objective optimization problems (MOPs). An important application of HW-NAS is real-time semantic segmentation, which plays a pivotal role in autonomous driving scenarios. The HW-NAS for real-time semantic segmentation inherently needs to balance multiple optimization objectives, including model accuracy, inference speed, and hardware-specific considerations. Despite its importance, benchmarks have yet to be developed to frame such a challenging task as multi-objective optimization. To bridge the gap, we introduce a tailored streamline to transform the task of HW-NAS for real-time semantic segmentation into standard MOPs. Building upon the streamline, we present a benchmark test suite, CitySeg/MOP, comprising fifteen MOPs derived from the Cityscapes dataset. The CitySeg/MOP test suite is integrated into the EvoXBench platform to provide seamless interfaces with various programming languages (e.g., Python and MATLAB) for instant fitness evaluations. We comprehensively assessed the CitySeg/MOP test suite on various multi-objective evolutionary algorithms, showcasing its versatility and practicality. Source codes are available at https://github.com/EMI-Group/evoxbench.
Code (1)
Tasks
Autonomous DrivingEvolutionary AlgorithmsHardware Aware Neural Architecture SearchNeural Architecture SearchReal-Time Semantic SegmentationSegmentationSemantic SegmentationSimilar Papers 제목 키워드 기반
A Scalable Benchmark Test Suite for Dynamic Multi-Objective Optimization with a Changing Number of Objectives
Dynamic multi-objective optimization with a changing number of objectives has recently attracted increasing attention due to its relevance to real-world problems whose evaluation criteria may evolve over time. However, e…
Using Well-Understood Single-Objective Functions in Multiobjective Black-Box Optimization Test Suites
Several test function suites are being used for numerical benchmarking of multiobjective optimization algorithms. While they have some desirable properties, like well-understood Pareto sets and Pareto fronts of various s…
BenchmarkingMultiobjective OptimizationSingle and Multi-Objective Optimization Benchmark Problems Focusing on Human-Powered Aircraft Design
The landscapes of real-world optimization problems can vary strongly depending on the application. In engineering design optimization, objective functions and constraints are often derived from governing equations, resul…
Evolutionary AlgorithmsCharacterization of Constrained Continuous Multiobjective Optimization Problems: A Feature Space Perspective
Despite the increasing interest in constrained multiobjective optimization in recent years, constrained multiobjective optimization problems (CMOPs) are still unsatisfactory understood and characterized. For this reason,…
BenchmarkingMultiobjective OptimizationBONO-Bench: A Comprehensive Test Suite for Bi-objective Numerical Optimization with Traceable Pareto Sets
The evaluation of heuristic optimizers on test problems, better known as \emph{benchmarking}, is a cornerstone of research in multi-objective optimization. However, most test problems used in benchmarking numerical multi…