Putting AI to the Test

Donor support helps a Navy veteran and his student team test whether AI can reduce the work and cost of hands-on cybersecurity training.

Justin Thorn came to the College of Business, Security and Intelligence Philanthropy Council with an itemized budget for $6,376 and two specific questions: Could artificial intelligence take over the grueling prep work behind hands-on cybersecurity training, and could relatively inexpensive systems do the job as well as larger commercial models?

The Navy veteran and Embry-Riddle cybersecurity student had already built a laboratory where students could practice attacking and defending computer networks. The catch was the maintenance. Preparing those exercises took instructors eight to 12 hours for every hour students spent in the lab. Impressed by his plan, the council awarded Thorn’s student-led team full funding.

A Safe Place to Break Things

Before arriving at Embry-Riddle, Thorn served in the U.S. Navy’s nuclear operations field. Now a cybersecurity student and captain of Embry-Riddle’s National Centers of Academic Excellence cybersecurity competition team, he sees a gap between understanding how a network works and knowing how to work with one.

“We’ve noticed that there is becoming more of a skills gap rather than a people gap in our industry,” Thorn told council members.

Students often grasp the theory behind defending a network yet still struggle with the physical equipment, connections and configurations that make one run—leaving employers to teach those practical skills on the job. Thorn wants students to graduate ready to work immediately, though cybersecurity poses a unique teaching hurdle: students need to practice both attacking and defending networks, and neither belongs on a live operational system.

“We need a safe place to break things,” he said.

For roughly $2,000, Thorn built a "purple lab" that combines the work of red teams, which simulate cyberattacks, and blue teams, which defend against them. Inside the simulated network, one student tries to penetrate the system while another detects the intrusion and works to stop it, allowing both to see the consequences of their choices and try again when they make mistakes.

Using two switches, two routers, four Raspberry Pi computers, and a host machine for additional virtual systems, Thorn developed 13 guided exercises that take students from basic cable connecting and labeling all the way to full attack-and-defense challenges.

Giving AI the Repetitive Work

Every one of those lab sessions requires instructors to configure equipment, build scenarios, fix errors and reset systems for the next group. With Philanthropy Council support, Thorn and two student colleagues are investigating whether specialized AI agents can reliably absorb that workload.

Their plan focuses on three practical roles. One agent would build and configure training exercises directly from an instructor’s written directions, eliminating the need to manually set up every component. Other agents would step in as virtual attackers or defenders when a human teammate is unavailable, while logging the sequence and timing of events so students can review their decisions and catch mistakes they missed in real time. Finally, a learning assistant trained on the lab’s exact hardware and configuration would help students troubleshoot why their specific setup isn't working, going beyond a general-purpose chatbot that only explains how a network should operate.

The goal is simple: give instructors more time to teach and students more time to practice. First, though, the technology has to prove it can handle the work.

Can AI Do the Job?

Rather than taking AI’s capabilities on faith, Thorn’s team will complete laboratory tasks manually, record the time required, and then have AI systems attempt the same tasks. By comparing speed, accuracy, and errors, the researchers will determine whether AI actually saves time and performs reliably.

The team will also pit AI models running locally on the college’s hardware against larger commercial models. Thorn says locally operated AI offers an important security advantage—keeping sensitive information within the institution rather than transmitting it to outside companies—while costing less to maintain and use. Whether those smaller systems are capable enough yet remains an open question.

“We do not know if local AI is able to run these labs properly, and that is why it is important to test them,” Thorn acknowledged.

The results could help other institutions build and maintain affordable cybersecurity laboratories without sacrificing the hands-on experience students need. Whether the systems succeed or fall short, Thorn and his colleagues will gain experience evaluating AI rather than simply accepting its claims.

The Price of an Answer

The $6,376 budget leaves little mystery about where the money will go: $1,000 is allocated for AI agents and related work, while the remaining $5,376 will pay Thorn and two student colleagues to conduct the research—giving them hands-on experience testing the technology they may encounter in their careers.

The award gives them the opportunity to test not just what AI can do, but where human judgment and intervention remain necessary.

“We need to figure out how to use it [AI] as a tool instead of accepting it as its own freestanding agent,” Thorn told council members.

His team will put that principle into practice, measuring the technology’s performance in a laboratory built for learning through trial and error. Thanks to the Philanthropy Council, they have exactly $6,376 to find out.