Evaluation of AI-Based Technology (C31c)
Building on previous talks at ICCC in Doha 2024 and Songdo 2025, which proposed a new AI security evaluation framework, the presenter is pleased to announce that the standardization project was approved last year supported by eight ISO National Bodies under ISO/IEC JTC 1/SC 27 (Information security, cybersecurity and privacy protection) /WG 3 (Security evaluation, testing and specification) and the project is titled ISO/IEC 25959: ‘Application of attack potential to deep learning-based technology’. This project is led by a team of eight editors from China, Korea, Japan, and the U.S., including the presenter as a Project Editor. This talk provides a pivotal update on the presenters’ progress. The presenters’ editorial team is currently developing a range of realistic attack scenarios, which are essential for assessing the actual risks facing AI systems. Since AI is susceptible to unique, evolving threats—such as adversarial examples, data poisoning, and model inversion—traditional cybersecurity measures are often insufficient. The presenter will also focus specifically on imminent threats targeting deep learning models, including LLMs, as well as the emerging landscape of agentic AI. Ultimately, the presenter’s goal is to provide robust security metrics that quantify the attacker potential required to execute these scenarios. This will facilitate a risk-based approach that aligns the necessary level of assurance with the specific attack scenarios that must be mitigated, as determined by the presenter’s security metrics. To conclude, The presenter will outline the presenters’ future roadmap for addressing these complex challenges. This includes the potential for establishing new projects to validate the presenter’s security metrics against a model Target of Evaluation (TOE) in practice.