Research

Trustworthy AI: Ensuring Safety in Cyber-Physical Systems

Achieving an AI-driven CPS system that is not only capable and efficient but also certifiably safe and trustworthy requires adhering to the NIST Trustworthy and Responsible AI1, which includes aspects such as validity and reliability, safety, security and resilience, accountability and transparency, and fairness. To this end, my research2 emphasizes the need for AI systems to be resilient against diverse environmental conditions, secure against adversarial threats, and transparent to the developers and intended users, particularly through three research thrusts:

  1. Trustworthiness-embedded CPS Development: The focus is to embed trustworthiness in every CPS development phase, from initial project scoping and design through to post-deployment. For instance, we ensure CPS sensors provide comprehensive environmental awareness thereby enhancing safety not only for the agent but also for all surrounding users3. We also incorporate fairness during the training of classification modules by emphasizing equal error reduction across all object classes4 and during testing and deployment by adopting an adaptable, risk-aware approach that carefully considers common and rare safety-critical scenarios to ensure safety for all5.
  2. Safe Online Adaptation: The goal is to implement robust mechanisms for continuous out-of-model-scope detections and retraining, alleviating the impracticality of covering every possible deployment scenario during the development phase. To this end, I developed a point-cloud annotation tool that allows operators to more easily identify rare objects and emphasize them in the training process6. I have also combined zero-shot classification for novel instances with data augmentation for efficient model retraining7.
  3. Certifiable Validation: Finally, to improve the efficiency of validating AI algorithms with already good performance (e.g., millions of miles of driving data may not sufficiently prove superior safety over humans due to the rarity of safety-critical incidents), I develop rare-event simulation techniques combining neural networks and importance sampling8 9 and analytically prove their theoretical guarantees10.

Looking ahead, the journey of integrating AI into CPS is both thrilling and challenging. My upcoming research represents bold strides into new territories, shaped by the dynamic nature of AI development. These endeavors promise to revolutionize safety, efficiency, and innovation in cyber-physical systems, marking an exciting applications-driven future for AI. I will continue to pursue the following research directions:

  1. Safety Certification throughout its lifecycle. I plan to extend the rare-event simulation to partially observable, multimodal, and mixed-autonomy AI applications and integrate multi-fidelity approaches11 to improve real-world testing efficiency.
  2. Multimodal CPS for expert assistants in critical tasks, such as dental work, surgery, or lab testing. My collaboration to push this direction forward will include colleagues from leading research institutions.
  3. AI for Safety and Sustainability applications which directly address the urgent need to certify ADAS, autonomous robots, and UAVs, as well as to safely and more efficiently perform mining and exploration of critical minerals and renewable energy using AI and unmanned systems. I am closely collaborating with Stanford MineralX and various mining, energy and exploration companies aiming to decarbonize the industry.

References

  1. NIST Trustworthy & Responsible Artificial Intelligence Resource Center. National Institute of Standards and Technology, U.S. Department of Commerce, https://airc.nist.gov/Home ↩

  2. Arief, Mansur. Certifiable Evaluation for Safe Intelligent Autonomy. Diss. Carnegie Mellon University, 2023. ↩

  3. Liu, Zuxin, Mansur Arief, and Ding Zhao. “Where Should We Place Lidars on the Autonomous Vehicle? An Optimal Design Approach.” 2019 International Conference on Robotics and Automation (ICRA). IEEE, 2019. ↩

  4. Arief, Hasan, Mansur Arief, Manoj Bhat, Ulf Geir Indahl, Håvard Tveite, and Ding Zhao. “Density-Adaptive Sampling for Heterogeneous Point Cloud Object Segmentation in Autonomous Vehicle Applications.” 2019 Conference on Computer Vision and Pattern Recognition (CVPR) Workshops. CVF, 2019. ↩

  5. Arief, Mansur, Peter Glynn, and Ding Zhao. “An Accelerated Approach to Safely and Efficiently Test Pre-production Autonomous Vehicles on Public Streets.” 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2018. ↩

  6. Arief, Hasan, Mansur Arief, Guilin Zhang, Zuxin Liu, Manoj Bhat, Ulf Geir Indahl, Håvard Tveite, and Ding Zhao. “SAnE: Smart Annotation and Evaluation Tools for Point Cloud Data.” IEEE Access 8 (2020): 131848-131858. ↩

  7. Gupta, Abhibha, Rully Hendrawan, and Mansur Arief. “Enhancing Visual Perception in Novel Environments via Incremental Data Augmentation Based on Style Transfer.” arXiv preprint arXiv:2309.08851, 2023. ↩

  8. Arief, Mansur, Zhepeng Cen, Zhenyuan Liu, Zhiyuan Huang, Bo Li, Henry Lam, and Ding Zhao. “Certifiable Evaluation for Autonomous Vehicle Perception Systems using Deep Importance Sampling (Deep IS).” 2022 25th International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2022. ↩

  9. Arief, Mansur, Zhepeng Cen, Huan Zhang, Henry Lam, Ding Zhao. “CERTIFY: Computationally Efficient Rare-failure Certification of Autonomous Vehicles.” Under review. ↩

  10. Arief, Mansur, Zhiyuan Huang, Guru Koushik Senthil Kumar, Yuanlu Bai, Shengyi He, Wenhao Ding, Henry Lam, and Ding Zhao. “Deep Probabilistic Accelerated Evaluation: A Robust Certifiable Rare-Event Simulation Methodology for Black-Box Safety-Critical Systems.” International Conference on Artificial Intelligence and Statistics (AISTATS). PMLR, 2021. ↩

  11. Huang, Zhiyuan, Mansur Arief, Henry Lam, and Ding Zhao. “Synthesis of Different Autonomous Vehicles Test Approaches.” 2018 21st International Conference on Intelligent Transportation Systems (ITSC). IEEE, 2018. ↩