live_tv
Livestream Starting Soon
00
Hours
:
00
Minutes
:
00
Seconds
Up next in 10
This episode dissects critical risks specific to Large Language Models (LLMs), focusing on vulnerabilities such as Prompt Injection and the potential for Sensitive Information Disclosure. It explores how CISOs must establish internal AI security standards and adopt a programmatic, offensive security approach using established governance frameworks like the NIST AI RMF and MITRE ATLAS. We discuss the essential role of robust governance, including mechanisms for establishing content provenance and maintaining information integrity against threats like Confabulation (Hallucinations) and data poisoning.
Sponsor:
www.cisomarketplace.services (http://www.cisomarketplace.services)
Show More Show Less View Video Transcript
0:00
Welcome to the deep dive. Today, our
0:02
mission is clear. We're taking this well
0:06
mountain of compliance papers, industry
0:08
reports, complex frameworks, everything
0:10
from NIST's AI RMF to MITERS AI and
0:14
boiling it down.
0:15
Yeah. Turning it into pure actionable
0:17
intelligence.
0:18
Exactly. We're deep diving into AI
0:20
security compliance and risk management.
0:23
And this is really tailored for you if
0:24
you're a CASO or a technical leader
0:26
trying to get your arms around this uh
0:28
exploding landscape.
0:30
And it's crucial we do this deep dive
0:31
because generative AI, GI, it's
0:34
introducing entirely new threat models.
0:36
It's not just, you know, more of the
0:37
same risk,
0:38
right?
0:38
It fundamentally changes the security
0:40
game. We need a strategy, something
0:42
agile that goes way beyond just
0:43
traditional network controls.
0:45
And the urgency, you just can't ignore
0:46
it. Looking at the source materials, I
0:48
mean, the alarm bells are ringing loud.
0:49
IBM's 2023 reports, they highlight that
0:53
82% 82% of all data breaches now involve
0:56
data stored in the cloud.
0:58
82%. And often the weigh-in isn't some
1:01
super complex hack.
1:02
No, it's simple misconfiguration.
1:04
Exactly. That number tells us the focus
1:06
has completely shifted. Perimeter
1:08
defense. That's well, it's becoming a
1:10
relic. Diligence in how we configure,
1:13
manage, monitor our internal cloud
1:15
assets, especially those running AI
1:18
workloads. That's the new mandate.
1:20
If you don't control your config, you've
1:21
lost
1:22
pretty much. You lose the battle right
1:23
there.
1:24
And navigating this level of complexity,
1:26
it really does require specialized
1:28
expertise. You need tools that
1:30
understand the specific risks around
1:33
LLM's cloudnative AI.
1:35
For sure,
1:35
which is why we really appreciate the
1:37
support of our sponsor SISO
1:38
marketplace.services. They help
1:40
organizations find exactly those kinds
1:42
of security resources needed for this uh
1:45
agile AIdriven risk landscape. a
1:48
necessary resource these days.
1:49
Okay, let's start where the pain often
1:51
is the cloud.
1:53
Deploying AI models on container
1:55
platforms, Kubernetes, Open Shift, you
1:57
name it. This whole shared
1:58
responsibility idea can become well bit
2:01
of a dangerous trap.
2:03
For our listeners, can you clarify
2:04
exactly where the cloud providers
2:05
responsibility stops and ours the
2:07
customers really begins with these AI
2:09
workloads?
2:10
Right. It's the classic distinction, but
2:12
honestly, the stakes are just higher
2:14
now. The cloud provider, they secure the
2:16
infrastructure of the cloud. Think
2:18
physical hardware, the hypervisor, the
2:20
big network pipes.
2:22
Okay. The base layer.
2:23
Yeah. But your organization, the
2:25
customer, you're on the hook for
2:27
absolutely everything in the cloud.
2:29
That sounds simple enough on paper, but
2:31
for a dynamically scaled AI service,
2:34
what does in the cloud really cover?
2:36
What's that list?
2:37
Oh, it's comprehensive. It's the
2:38
application workloads themselves
2:40
obviously the integrity of your
2:42
container images the runtime data
2:44
protection measures but for CSOS the
2:47
absolute non-negotiables identity and
2:49
access management AM for your models and
2:51
code and the configuration of the
2:53
container orchestration data plane
2:55
that's critical
2:56
container orchestration data plane
2:58
that's pretty dense can we break that
2:59
down what specifically needs locking
3:01
down there
3:02
we're talking about the network traffic
3:03
rules things managed by tools like CNI
3:06
or QRoxy and Kubernetes etes. We're
3:08
talking network policies which
3:10
containers can talk to your databases
3:11
for instance. Secrets management. Think
3:14
about it. If your AI model is in a
3:16
Kubernetes cluster and that cluster's
3:18
internal networking is wide open because
3:20
of a misconfiguration, well, you've just
3:22
created an open highway for an attacker
3:24
if they breach even one container.
3:27
Lateral movement becomes easy.
3:29
And we've seen the consequences. The
3:31
sources remind us about Capital One huge
3:33
fines, right? All stemming from a flawed
3:35
web firewall config that let an attacker
3:38
grab customer records.
3:39
Exactly. A classic costly mistake.
3:41
So if that was the result of a
3:43
traditional firewall lapse, what happens
3:45
if we apply that same kind of lapse to
3:48
say an overly permissive AM role given
3:51
to a Gen AI agent?
3:52
Oh, the fallout gets magnified like
3:54
exponentially. A configuration failure
3:56
there, maybe an IMM role letting an LLM
3:59
agent read and write to a critical S3
4:01
bucket it shouldn't touch.
4:02
Disaster. It can lead to instant data
4:04
exfiltration and that lapse immediately
4:06
triggers major compliance breaches.
4:08
We're talking GDPR fines potentially
4:09
hitting 20 million or 4% of global
4:11
turnover. IPA violations. The security
4:14
team's focus has to shift from just the
4:15
perimeter to meticulous constant
4:17
diligence on your side of that shared
4:19
model.
4:20
Okay, let's unpack the really novel
4:22
threats then. Traditional security
4:24
understands network flaws, application
4:26
bugs, but GI throws these new sort of
4:30
non-conventional attack vectors into the
4:32
mix. What are maybe the top most
4:34
surprising or insidious ones that need
4:37
custom defenses beyond just a standard
4:39
W?
4:40
Yeah, we have to shift our thinking.
4:41
It's about semantic manipulation. Now,
4:44
NIST's generative AI profile, it clearly
4:46
outlines risks where the AI itself is
4:48
the vulnerability. The first one and
4:50
maybe the most immediate prompt
4:52
injection and goal hijacking.
4:54
The AI age SQL injection as you put it.
4:57
Can you break down the mechanics? How
4:59
does simple text suddenly become a
5:01
malicious command?
5:02
It's because the model is fundamentally
5:04
designed to follow instructions. That's
5:05
its job. So attackers craft input a
5:07
malicious instruction basically that the
5:09
model sees as a priority.
5:10
Okay.
5:10
It then overrides its built-in safety
5:12
instructions, its system prompt, its
5:14
guard rails. A common goal is tricking
5:16
the model into revealing protected info.
5:18
maybe the underlying training data, code
5:19
snippets, or even just the persona it's
5:21
supposed to maintain.
5:22
And it's not just about someone typing a
5:25
nasty request directly into the chatbot,
5:26
is it? There are indirect methods, too.
5:29
Absolutely. And that's where it gets
5:30
really insidious. The critical threat
5:32
here is indirect prompt injection,
5:35
especially if you're using retrieval
5:36
augmented generation or rag.
5:38
Rag. Yeah. Where it pulls in external
5:40
data.
5:41
Exactly. So, the malicious commands
5:42
aren't typed by the user. They're
5:44
hidden, embedded in the data the model
5:46
retrieves. could be a web page, a
5:49
company wiki doc, a knowledgebased
5:50
article. The model reads this text,
5:54
thinks it's just necessary context,
5:56
and executes the hidden command
5:57
without the user having a clue. It
5:59
basically poisons the model's working
6:01
environment from the inside.
6:02
Okay, here's one that's maybe less about
6:04
malice and more about the tech itself.
6:06
Confabulation,
6:07
hallucinations.
6:09
Why is this a security and compliance
6:11
risk, not just an accuracy headache?
6:13
It's the confidence. LLMs predict the
6:16
next word based on statistics, right?
6:18
Not actual facts. So, they can generate
6:20
outputs that sound incredibly
6:21
convincing, authoritative even, but are
6:24
just plain wrong. The risk from a
6:26
compliance and security angle comes down
6:28
to automation bias.
6:30
Meaning, users just trust the
6:32
sophisticated output too much. They
6:34
assume it's correct because it sounds
6:36
good. Imagine an internal audit agent
6:39
powered by an LLM that just makes up a
6:41
policy violation or cites regulations
6:43
that don't exist.
6:44
Right?
6:44
If a human auditor automatically trusts
6:46
that output without verification that
6:49
leads directly to bad business
6:50
decisions, financial loss, or in
6:52
regulated industries, huge liability.
6:55
Think incorrect legal strategies, wrong
6:57
medical assessments. It's serious. And
7:00
the third big vector, this one feels
7:03
maybe more familiar from a traditional
7:05
security view, but devastating in AI,
7:07
data leakage and model exposure.
7:09
Yeah, this is attackers targeting
7:10
misconfigurations again. But the goal is
7:13
exfiltrating training data credentials
7:15
and crucially the intellectual property
7:17
baked into the model weights themselves.
7:20
The hugging face incident where people
7:21
worried private API keys might have been
7:23
exposed
7:24
shows the urgency.
7:25
Exactly. When someone steals your
7:26
uniquely trained model, they get
7:29
potentially years of your R&D advantage,
7:31
or they can use it to create a
7:33
functional copy, maybe to bypass your
7:35
defenses somewhere else.
7:36
Now, before we get to mitigation, there
7:38
was a fascinating bit in the sources
7:39
about environmental risk. It's not data
7:42
theft, more governance. How should a
7:44
CISO even think about folding that into
7:47
their risk assessment? It's often
7:49
overlooked, but it fits squarely under
7:51
the NIST RMF's govern function,
7:53
especially around resource use and
7:55
increasingly ESG compliance. Training
7:58
just one large transformer LLM takes
8:00
immense compute power.
8:02
Right. The sources mentioned emissions
8:03
like 300 roundtrip flights and SF to New
8:06
York,
8:07
something like that. Yeah, massive. This
8:08
huge compute dependency means that the
8:10
availability and the cost management of
8:12
your AI pipeline become critical
8:14
governance risks. Over reliance on
8:16
certain proprietary models or just
8:18
poorly optimized training runs can
8:20
introduce really significant operational
8:22
overhead and potential compliance issues
8:25
related to resource consumption down the
8:26
line. It's another layer and mitigating
8:30
all these complex layered threats, some
8:32
abstract, some very concrete elite, it
8:35
definitely requires moving past ad hoc
8:37
fixes, you need integrated strategies.
8:40
No question.
8:40
Which again highlights why having access
8:43
to those niche security services, the
8:44
right tools, the expertise in things
8:46
like prompt validation, AI governance,
8:49
it's vital. And that's precisely why we
8:51
value our sponsor CISA
8:53
marketplace.services services for
8:54
connecting folks with that specialized
8:56
talent needed for robust, compliant AI
8:58
defenses.
8:58
Definitely a key enabler.
9:00
So, we know the risks are high, they're
9:01
complex. How do organizations actually
9:03
build resilience? What are the practical
9:05
steps the frameworks CSOS should be
9:07
using to formalize their AI security
9:09
program?
9:10
We absolutely have to shift to a
9:11
proactive compliance-driven mindset. And
9:13
that means centering on established
9:15
frameworks. The big one right now is the
9:17
NIST AI riskmanagement framework, the AI
9:20
RMF. It's technically voluntary, but
9:22
it's rapidly becoming the deacto global
9:25
standard for establishing AI
9:26
trustworthiness. It's built around four
9:28
core functions. Okay, let's break this
9:30
down. The first one, govern. How does
9:32
that translate into a CISO's action
9:35
plan? What do you actually do?
9:36
Govern is about setting the culture, the
9:38
structure for the CISO. This means
9:40
defining clear crossfunctional roles for
9:42
AI risk. Who owns what? Establishing
9:45
clear lines of accountability. mandating
9:48
transparent documentation not just about
9:50
model performance but about its ethical
9:52
boundaries, its safety limits. It's
9:55
about elevating AI risk from just a tech
9:57
team problem to a board level concern.
10:00
Moving it up the chain. Okay. Once
10:01
you've governed the approach, you map.
10:03
What does effective mapping look like
10:05
for AI?
10:05
Mapping is all about context. You need
10:08
to assess the intended use case. What's
10:10
this AI actually for? What are the data
10:12
sources? The deployment environment and
10:14
crucially, what are the potential impact
10:16
services? Is it internal low stakes or
10:19
external high stakes like loan approvals
10:21
or medical diagnosis?
10:22
Big difference.
10:23
Huge. And you map the risk profile for
10:25
each specific use case against your
10:27
existing regulatory obligations. A pay,
10:30
CCPA, GDPR, whatever applies.
10:32
Then comes measure and manage,
10:33
right? Measure is the technical deep
10:35
dive. This is where you do model
10:36
vulnerability scanning, adversarial
10:38
testing, trying to break it, evaluating
10:40
risks like how susceptible it is to
10:42
prompt injection. You're putting a
10:44
number on the risk, quantifying it.
10:46
Okay,
10:47
manage the action part taking those
10:49
measured risks, prioritizing them and
10:51
then implementing controls, input output
10:53
filtering, rate limiting, anomaly
10:55
detection and critically monitoring the
10:58
model continuously for performance drift
11:00
or any signs of attack or misuse.
11:02
Now for the tech teams actually
11:04
implementing these controls, the sources
11:06
also mentioned the MITER safe AI
11:08
framework.
11:09
How does that fit in with the NIST RMF?
11:12
Think of MITER safe AI as more tactical.
11:14
It builds on the NIST RMF philosophy but
11:17
integrates it with MITER Atlas's threat
11:19
informed defense approach. It helps by
11:21
dividing the whole AI system into four
11:23
security focus areas. Yeah. The
11:25
environment it runs in, the AI platform
11:26
and tools, the AI models themselves, and
11:29
the AI data.
11:30
So it helps organize the defense.
11:31
Exactly. It ensures security teams
11:33
aren't just looking at the network edge,
11:34
but applying specific controls within
11:36
the model development and deployment
11:38
life cycle itself.
11:39
Which brings us to MLC tops. It's kind
11:41
of a buzzword, but what does it actually
11:43
demand in practice?
11:44
MLC cops basically means security isn't
11:47
an afterthought, not just a QA check at
11:49
the end. It mandates things like threat
11:52
modeling before you even start
11:53
developing a model. Thinking about how
11:55
it could be attacked from day one.
11:57
Sift left.
11:58
Totally. It means incorporating model
12:00
vulnerability scanning into your CI/CD
12:03
pipelines just like you do for code and
12:05
doing serious adversarial testing,
12:07
simulating prompt injections, data
12:09
poisoning attempts before you push to
12:11
production. It's about embedding
12:13
security deep into the data science and
12:15
ML engineering workflow.
12:16
Okay, based on everything we've
12:18
discussed from the sources, what are
12:20
maybe the three most actionable like
12:21
right now mitigation strategies our
12:23
listeners should focus on?
12:24
All right, three key things. First,
12:26
fundamentally rethink layered defenses,
12:28
your traditional firewall. It's not
12:30
enough for semantic attacks like prompt
12:32
injection. You need more advanced
12:34
contextaware input validation.
12:37
What does that look like?
12:38
Often it means using things like sidecar
12:41
containers and Kubernetes or specialized
12:43
microservices that don't just look at
12:45
the payload content, but scrutinize the
12:47
intent and structure of the input.
12:49
Applying multiple checks before that
12:51
prompt even gets near the core LLM.
12:53
Makes sense.
12:54
Yeah. Second, given the Capital One
12:56
disaster,
12:56
least privilege. It cannot be stressed
12:58
enough. It needs rigorous
13:00
implementation, especially for AI agents
13:02
and plugins. An LLM agent should have
13:05
only the bare minimum AM permissions
13:07
needed for its specific job. If it
13:09
summarizes documents, it absolutely
13:12
should not have rights to delete user
13:13
records.
13:14
The granular controls,
13:15
very granular AM conditions. That's how
13:17
you limit the blast radius when not if
13:19
an injection attack succeeds. And
13:21
finally, maybe the most sophisticated
13:23
tool is still us.
13:24
Absolutely. The human in the loop. It is
13:26
simply non-negotiable for critical
13:28
applications. LLMs are probabistic. They
13:31
confabulate. They have confidence bias.
13:33
You must have human experts reviewing
13:35
and validating outputs.
13:36
Especially for important decisions,
13:38
especially for critical decisions or
13:39
even just low confidence results flagged
13:41
by the system itself. Relying solely on
13:44
the AI, that automation bias again is
13:46
just the fastest route to systemic
13:48
failure and major compliance headaches.
13:51
So, if we boil this whole deep dive
13:53
down, AI security isn't just a tech
13:55
problem anymore. It's fundamentally a
13:57
board level governance and compliance
13:59
challenge. Your organization's future
14:01
financially and regulatory wise, it
14:03
doesn't hinge on the cloud provider's
14:05
security nearly as much as it hinges on
14:07
your diligence,
14:09
your side of the shared responsibility
14:11
model. You have to prove effective
14:13
configuration, ongoing risk management.
14:15
That's a perfect summary. So, we'll
14:17
leave you, the listener, with one final
14:18
provocative thought. This draws on a
14:20
specific risk highlighted in the sources
14:22
that isn't strictly security but it's
14:24
related. The risk of harmful bias and
14:26
algorithmic homogenization.
14:28
Right? This is the potential danger of
14:29
creating what some call algorithmic
14:31
monocultures. If say every organization
14:34
starts relying heavily on the same
14:36
handful of big foundation LLMs
14:38
which is happening
14:38
which is happening
14:40
well any underlying biases in those
14:43
models just get amplified and spread
14:45
across entire industries. And beyond
14:47
bias, just relying on the same model
14:49
over and over can lead to a kind of
14:51
functional stagnation. Outputs become
14:54
uniform, less diverse. Creative spark
14:56
gets sacrificed for efficient,
14:58
predictable, but maybe bland results.
15:01
So here's something to consider for your
15:03
strategic road map. If your organization
15:05
is leaning heavily on a standard
15:07
foundation LLM for creative work, for
15:09
research, for critical decision-making,
15:12
how are you actually measuring whether
15:13
you might be inadvertently perpetuating
15:15
systemic biases, or perhaps even
15:18
sacrificing your competitive edge and
15:20
creative diversity just for the sake of
15:22
efficient but ultimately uniform output?
15:25
A really vital question.
15:26
Definitely something to chew on this
15:27
week. Thank you for joining us for this
15:29
deep dive into AI risk management. And
15:31
once again, our sincere thanks to our
15:32
sponsor, SISO Marketplace. If you need
15:35
that specialized talent, those specific
15:37
tools to tackle these GI security and
15:39
governance challenges, SISO
15:40
marketplace.services is the place to
15:42
look.
15:42
Check them out. We'll see you next time
15:44
on the deep dive.