LibMOON: A Gradient-based MultiObjective OptimizatioN Library in PyTorch
Multiobjective optimization problems (MOPs) are prevalent in machine learning, with applications in multi-task learning, learning under fairness or robustness constraints, etc. Instead of reducing multiple objective functions into a scalar objective, MOPs aim to optimize for the so-called Pareto optimality or Pareto set learning, which involves optimizing more than one objective function simultaneously, over models with thousands / millions of parameters. Existing benchmark libraries for MOPs mainly focus on evolutionary algorithms, most of which are zeroth-order / meta-heuristic methods that do not effectively utilize higher-order information from objectives and cannot scale to large-scale models with thousands / millions of parameters. In light of the above gap, this paper introduces LibMOON, the first multiobjective optimization library that supports state-of-the-art gradient-based methods, provides a fair benchmark, and is open-sourced for the community.
Code (1)
Tasks
Evolutionary AlgorithmsFairnessMultiobjective OptimizationMulti-Task LearningMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
PyTorch Tabular: A Framework for Deep Learning with Tabular Data
In spite of showing unreasonable effectiveness in modalities like Text and Image, Deep Learning has always lagged Gradient Boosting in tabular data - both in popularity and performance. But recently there have been newer…
Deep LearningASDL: A Unified Interface for Gradient Preconditioning in PyTorch
Gradient preconditioning is a key technique to integrate the second-order information into gradients for improving and extending gradient-based learning algorithms. In deep learning, stochasticity, nonconvexity, and high…
torchgpipe: On-the-fly Pipeline Parallelism for Training Giant Models
We design and implement a ready-to-use library in PyTorch for performing micro-batch pipeline parallelism with checkpointing proposed by GPipe (Huang et al., 2019). In particular, we develop a set of design components to…
UncertaintyPlayground: A Fast and Simplified Python Library for Uncertainty Estimation
This paper introduces UncertaintyPlayground, a Python library built on PyTorch and GPyTorch for uncertainty estimation in supervised learning tasks. The library offers fast training for Gaussian and multi-modal outcome d…
CPUGPUPrediction IntervalsTorchOpt: An Efficient Library for Differentiable Optimization
Recent years have witnessed the booming of various differentiable optimization algorithms. These algorithms exhibit different execution patterns, and their execution needs massive computational resources that go beyond a…
CPUGPU