Problem: An unexpected AI failure can have severe consequences, affecting human lives in various ways, including safety, productivity, trust in AI, and ethical considerations. Understanding AI's vulnerabilities is an essential task. However, it is generally considered challenging and resource-intensive (e.g., time, expertise, data) for humans — specifically Machine Learning engineers who are proficient in and responsible for this task.

Vision & Approach: To support ML engineers' workflows, we aim to develop novel human-AI collaboration designs that enable ML engineers to investigate and fix AI vulnerabilities more efficiently and effectively. We identified two key methodological pillars: (1) Scalable Human-In-The-Loop — maintaining a reasonable level of human input even when tasks are overwhelming; and (2) Actionable Interactive Model-Steering — enabling humans to convert insights into direct model updates via intuitive interactions.

Impact: Our solution can enhance human-AI interaction, empowering ML engineers to work across diverse datasets — images, time series, text, and beyond — with more reliable, trustworthy, and controllable AI.