"Assessing the Effectiveness of AI-Generated Software in Withstanding C" by Dominic Niceforo and Haydar Cukurtepe
 

Level of Education of Students Involved

Undergraduate

Faculty Sponsor

Haydar Çukurtepe

College

College of Arts & Sciences (CAS)

Discipline(s)

Computer Science, Software Security

ORCID Identifier(s)

0009-0001-9861-9519, 0000-0002-4670-4877

Presentation Type

Poster Presentation

Symposium Date

Spring 4-24-2025

Abstract

This project introduces a streamlined framework for evaluating the security of Java applications generated by large language models (LLMs). It integrates OWASP-supported tools with the NIST Risk Management Framework to evaluate code resilience against cyber threats and align relevant assessments with industry standards. The Java applications that are generated by LLMs undergo a three-phase analysis: SpotBugs with FindSecBugs to detect static vulnerabilities before runtime, OWASP Dependency Check to scan for known vulnerabilities in third-party libraries, and OWASP ZAP to simulate real-world attacks through dynamic testing. Identified vulnerabilities from these three tests are then passed through NIST to be standardized and assigned a risk level based on potential sources, likelihood, and impact of the threat. The impact of prompt design is explored by testing various AI models with varying prompts based on their emphasis on secure coding. The process outlined in the study provides a consistent and reproducible method for measuring AI-generated code against existing software development practices.

Biographical Information about Author(s)

Dominic Niceforo is a senior computer science student at Valparaiso University, focusing on software security and artificial intelligence.

This project was completed under the guidance of Professor Haydar Çukurtepe, who teaches in the Department of Computing and Information Sciences.

Share

COinS