Using Local LLMs and RAG to Support Secure Cybersecurity Certification Workflows (A22b)

Public generative AI services may not be suitable for cybersecurity certification work involving sensitive policies, control documents, assessment evidence, and internal information. Running a large language model locally provides greater control over the data, but setting up and adapting a local model can appear technically difficult.

This session presents a simple, practical workflow for downloading and configuring a local LLM platform with Retrieval-Augmented Generation (RAG). Rather than retraining the model, users can add their own cybersecurity certification and governance documents to a local knowledge base, allowing the model to retrieve relevant information when answering questions.

 

Through a step-by-step demonstration, attendees will see how to install the platform, select and run a local model, configure RAG, upload reference documents, and begin querying the material. The session will also cover common setup challenges and practical considerations when working with local models.

 

Attendees will leave with an accessible method for experimenting with generative AI in cybersecurity certification workflows while keeping source documents within their own environment and avoiding the complexity of training a model from scratch.