Automated Dependence Plots
In practical applications of machine learning, it is necessary to look beyond standard metrics such as test accuracy in order to validate various qualitative properties of a model. Partial dependence plots (PDP), including instance-specific PDPs (i.e., ICE plots), have been widely used as a visual tool to understand or validate a model. Yet, current PDPs suffer from two main drawbacks: (1) a user must manually sort or select interesting plots, and (2) PDPs are usually limited to plots along a single feature. To address these drawbacks, we formalize a method for automating the selection of interesting PDPs and extend PDPs beyond showing single features to show the model response along arbitrary directions, for example in raw feature space or a latent space arising from some generative model. We demonstrate the usefulness of our automated dependence plots (ADP) across multiple use-cases and datasets including model selection, bias detection, understanding out-of-sample behavior, and exploring the latent space of a generative model.
Code (2)
Tasks
Bias DetectionModel SelectionSelection biasMethods 이 논문이 사용한 방법론
Similar Papers 제목 키워드 기반
Towards Explaining Hyperparameter Optimization via Partial Dependence Plots
Automated hyperparameter optimization (HPO) can support practitioners to obtain peak performance in machine learning models. However, there is often a lack of valuable insights into the effects of different hyperparamete…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationInterpretable Machine LearningWhy You Should Not Trust Interpretations in Machine Learning: Adversarial Attacks on Partial Dependence Plots
The adoption of artificial intelligence (AI) across industries has led to the widespread use of complex black-box models and interpretation tools for decision making. This paper proposes an adversarial framework to uncov…
Decision MakingManagementExplaining Hyperparameter Optimization via Partial Dependence Plots
Automated hyperparameter optimization (HPO) can support practitioners to obtain peak performance in machine learning models. However, there is often a lack of valuable insights into the effects of different hyperparamete…
Bayesian OptimizationBIG-bench Machine LearningHyperparameter OptimizationInterpretable Machine LearningScatteract: Automated extraction of data from scatter plots
Charts are an excellent way to convey patterns and trends in data, but they do not facilitate further modeling of the data or close inspection of individual data points. We present a fully automated system for extracting…
Optical Character RecognitionOptical Character Recognition (OCR)Leveraging Vision Capabilities of Multimodal LLMs for Automated Data Extraction from Plots
Automated data extraction from research texts has been steadily improving, with the emergence of large language models (LLMs) accelerating progress even further. Extracting data from plots in research papers, however, ha…