SAN ANTONIO, Texas — The University of Texas at San Antonio is using a $500,000 National Science Foundation grant to train students to identify and fix security vulnerabilities in AI-generated code as concerns grow over the risks of autonomous AI systems.
Nishant Vishwamitra, an assistant professor in UTSA’s Department of Information Systems and Cybersecurity, leads the grant-funded project, which aims to prepare students to identify security risks before AI-generated code reaches critical systems.
The program uses OpenSecCoder, a tool that allows students to work with AI-generated software and examine potential vulnerabilities in a controlled environment.
“OpenSecCoder has these sandbox containers where students can actually observe and execute attacks and also defenses in a safe environment,” Vishwamitra told Texas Public Radio.
Preparing for agentic AI
Traditional AI chatbots generally respond to user requests, while agentic AI can perform multiple tasks on its own, including creating files, installing software and running commands.
Vishwamitra said autonomous systems could introduce vulnerabilities into software that later reaches healthcare, defense and other critical applications.
“AI could introduce vulnerabilities that could later find its way into healthcare or even defense applications,” he said. “These vulnerabilities can go unchecked and then attackers can exploit these vulnerabilities to compromise these critical systems.”
The program will place students in eight separate laboratory environments where they can analyze malicious code and study potential defenses.
Students will conduct simulated attacks, observe their effects and examine ways to identify and correct vulnerabilities in AI-generated code.
Training future developers
Vishwamitra said the project aims to address security problems at the coding stage rather than after vulnerable software reaches critical systems.
“Once these vulnerable codes find their way into these critical applications, this can be easily exploited by attackers,” he said.
The project aims to teach future developers to follow secure coding practices and identify vulnerabilities before attackers can exploit them.
The OpenSecCoder program expects to reach about 600 undergraduate and 400 graduate students each year.
The project comes amid growing concerns about the ability of agentic AI systems to perform actions without direct human intervention.
A recent incident involving an AI agent and an Australian national healthcare database has further fueled those concerns. According to the Australian government, the agent accessed public and non-public files.
AI industry leaders, including OpenAI CEO Sam Altman, have also called for caution as companies continue developing increasingly autonomous AI systems.