 IBM's AutoAI provides end-to-end AutoML, including data preparation and model selection, to optimize for some notion of predictive performance, which significantly reduces the time taken by data scientists to generate prediction pipelines. This IBM research demo showcases a new capability that can incorporate problem or domain-specific stakeholder constraints in the optimization for end-to-end AutoML. We will demonstrate this capability with a credit application problem. This German credit dataset contains information, features regarding applicants, and we wish to predict if the applicants have good or bad credit. This data is historical with ground truth available, and we wish to automatically learn a machine learning prediction pipeline to make predictions on future applicants. This is posed as a binary classification problem, and we can optimize for the area under the ROC curve. At this point, the user can click start and kick off the optimization with three clicks, data upload, target column selection, and the start button. At this point, the problem can be customized. We can select a different metric to optimize for, such as accuracy or F1 score. Furthermore, we can specify stakeholder constraints. Here, we provide a canned set of constraints, but we will demonstrate later how one can even specify custom constraints. First, we specify a couple of AI fairness constraints. The parity difference constraint makes sure in this application that the favorable outcome rate, the prediction rate for good credit, is close between different groups. Here, we are grouping the data, the population based on gender, but you can choose other factors such as age or location. This specific constraint makes sure that we do not predict good credit at a high rate, say 30% for males, but at a low 5% for females. The other fairness metric error rate difference makes sure that the predictions have close predictive performance between different groups, so that it is not the case that the error rate is 10% for males and 40% for females. Then we can specify other constraints on the inference time and the memory overhead for the prediction pipeline. False positive rate is another business constraint. There might be a regulatory constraint on this too, since a false positive implies we are predicting bad credit for someone who has good credit, subsequently denying the loan application. It can be deemed both discriminatory and lost business for a loan vendor. If the user has a constraint with respect to their own custom metrics such as next season's revenue, they can define them and AutoAI can incorporate them in the optimization as well. Given these specified constraints, we can execute the optimization. On this page, given the parameters of the problem up here, we see that around 20 pipelines were tried, of which only 5 satisfied all constraints simultaneously. Here we show the different modeling operators used, along with the number of pipelines containing these modeling operators that satisfy or violate some constraints. And down here, we can see the leaderboard view. We see different pipelines, such as an XGBoost classifier. For this pipeline, we can view different predictive performance metrics, the ROC curve, and the feature importances. Here is another pipeline with a robust scaling and a truncated SVD, a singular value decomposition preceding the XGBoost classifier. There is a column for the metric corresponding to the objective in the optimization, the ROC curve, the area under the ROC curve, and a separate column for the metrics corresponding to the constraints. In this case, the inference time constraints with the green arrow and the red arrow indicating constraint satisfaction and constraint violation, respectively, for this particular constraint. We can use the dropdown here to view the metrics corresponding to other constraints, such as the false positive rate. Beyond these views, we also developed a novel visualization that allows us to both compare different pipelines in terms of the metrics, but also view the internals of the optimization trajectory, the conditional parallel coordinates visualization, or the CPC visualization. There are three main sections. This left section shows the different modeling, data preparation, and featureization operators used by the different pipelines. The middle section shows the different constraints, and the right section shows a set of predictive performance metrics beyond the one we optimized for. Each line corresponds to a single pipeline. This one shows a pipeline with a scaling operator and a transformer using sampling with radial basis functions before a quadratic discriminant analysis modeling estimator. Then we can see which of the constraints this pipeline satisfies or violates. The green section in the vertical bar implies constraint satisfaction, while the red section implies constraint violation. Finally, we see the different performance metrics. Furthermore, for expert users, they can even dig into specific operators and see the different configurations, the hyperparameters that were considered in the optimization. This demonstrates the conditional nature of the conditional parallel coordinates visualization. AutoAI can easily generate dozens or hundreds of pipelines, whereas a human data scientist normally generates a couple of pipelines for consideration. That is why there is a lot packed in the CPC visualization to give one the complete information regarding the optimization. And it is particularly useful for data scientists to make a final selection of a pipeline generated by AutoAI. Finally, the same functionality can be obtained via a Jupyter notebook. With this programmatic interface, one can define custom constraints using a simple API. Thank you.