AI audio deepfakes are leading new ai-impersonation scams

But I just got in a car crash on I-77

I swear I'm totally fine, but I don't have my wallet with me. Can you please, please, please Venmo me $300 so I can pay this tow truck guy

Definitely AI once again. Scammers have always tried to take our hard-earned money and personal information

but Genitive AI is making it even easier for them. Now, with the click of a button, they can create pretty decent content that looks and sounds just like us

This is called AI impersonation, and it's a growing problem for everyone

AI makes it easier to make fake content so you can make more of it and distribute it widely

We like to call it the three Vs. We've seen the volume increase

We've seen the velocity increase. And then we've seen the vectors increase

So what that means is that whereas before it would take a threat actor a certain amount of time as well as resources in order to create that deepfake, it doesn't take that same amount of time anymore

It's done at much faster clips, so the volume increases. And then I said that we've seen the vectors increase, whereas before it was primarily video or focused on audio

we've seen it across multiple channels, whether it's video, audio, email. And so we've seen those three V's increase

Thanks to AI, it's easier than ever to make deep fakes. Celebrities, public figures, and politicians are all high-profile deep fake targets

But the goal isn't always to make it seem like they said or did something. The goal is to get us to believe that they did, even if it's not true

That ability to influence our beliefs matters a lot, especially in politics

During the 2024 primary presidential election, thousands of voters in New Hampshire got a

surprise phone call from Joe Biden the night before the election, urging them not to vote

You're a bunch of malarkey. You know the value of voting Democratic when our votes count

It's important that you save your vote for the November election. Except, of course, it wasn't Joe Biden

It was an AI deepfake. But the voice in the calls sounded like him

Voters couldn't tell the difference. The Biden campaign tried to let people know that the call was fake, but the damage was already done

The ability to influence is what matters in a campaign, right? That's what campaigns are doing, right

They're trying to influence people to support them, right? They're trying to get people to rally people

And so I think they may not have the tools for cybersecurity, but they're also out there in the world using all the platforms that the bad actors use

in order to reach the same group of people that the bad actors might be trying to reach

The goal is to erode trust and sow chaos. And it isn't just during election season

In July Senator Mitch McConnell office released a picture of him during his recent hospitalization Users on social media immediately began questioning whether it was real Many folks shared the theory that the photo was either old or edited with AI tools

Digital forensic experts found no evidence that it was an old photo or edited with AI

But we live in a world now where our first thought is that might not be real

When that photo was released, if that photo had been released two years ago

Two years ago, it would have taken me probably a good seven or eight hours to actually doctor that photo and to actually create different variations of that photo

It would have taken at least seven to eight hours. Most people don't have seven to eight hours to doctor a photo or to create variations of that photo

But now with generative AI, all I need to do is copy that photo, drop it into an app, give it some plain language commands

And within, you know, 30 seconds, I can have 10 different versions of that same photo

And then I can send those 10 different versions to my friends. So that's the impact that generative AI has had on causing us to question what we see

It's that speed and ease that's leading to a new trend in fraud. Bad actors are using AI to manipulate your emotions and personally target you

They do this by sending a message with a sense of urgency. So you don't pause to think about a reply

That could be through fear of safety, fear of losing money, panic about something that doesn't exist, or maybe it's just straight up rage bait

The goal is to get your emotions to cloud your judgment. So one of the best ways you can fight back is to take a breath, step back, and think before you do anything rash

And ask yourself, why is this message being sent to me? But sometimes that's easier said than done

The point of this media is a manipulation of your emotions, right

So you need to kind of be in touch with that and not spend a lot of time trying to figure out like, wait, is that, is Michael real in that picture or is he not real in that picture

Because it's getting extremely hard to tell. We expect us to see, you know, the volume increase significantly, and it's only going to get worse

But more concerning, again, to Michael's point, is the hyper personalization that we're going to see

Whereas generative AI, it's taken days and weeks and converted that into minutes and seconds

And so now I don't need to do a bunch of research on Caitlin in order to target her

I could, you know, go to some app and have it write either an email or have it develop

some synthetic voice and call you based on open source intelligence it can gather on you

That's the difference. This is an AI clone of my voice that was made by Doppel

I decided to send it to my boss to see if he could tell the difference between me and AI Hi this is Caitlin Shidrowi I a reporter with CNET I reaching out because I wrapping up a story on AI impersonation and just need one quick favor Could you send me the contact information

for the campaign spokesperson we interviewed yesterday? I need to verify a few more details

from my story in the next two hours or it won't run at all. Thank you. Hi, this is Caitlin Shidrowi

I'm a reporter with CNET. I'm reaching out because I'm wrapping up a story on AI impersonation and

just need one quick favor. Could you send me the contact information for the campaign spokesperson

we interviewed yesterday? I need to verify a few more details for my story in the next two hours

or it won't run at all. Thank you. Hey, this is John Reed. I'm a managing editor at CNET. I'm

Caitlin's boss. And I think version two is fake, is AI generated. A couple reasons for that

first one in version one you can detect kind of a faint room tone in the background that's pretty

much absent in version two sounds like it was actually recorded with a microphone in a room

with like a fan blowing in the background you can only really tell that when you're like you've got

the volume turned up second version two has this kind of like blend that sounds like it's multiple

voices blended together that i'd notice in a lot of like ai generated videos and sounds so to show

you just how convincing this AI impersonation tech is, we're going to do a little experiment

So with me today is the one person who knows me better than anyone else in the world, my sister

Nicole. Hello. So today we're doing a kind of audio blind taste test. Earlier this week, I cloned

my voice using Eleven Labs AI. For only $6 a month, I was able to make a scarily realistic

version of my voice. So I gave it some scripts. I wrote out some messages that sound like they

would be voicemails or voice notes that I would leave for my friends and family

I also recorded myself in my human voice saying these messages. And our challenge today is to see

if you can tell the difference between what is AI and what is me. Hi there, this is Caitlin. I'm a

patient at your office. I'm calling to follow up from my doctor's appointment last week, and I need

you to send me my complete medical record. Please let me know what the next steps are. Thanks

I think definitely AI. Really? 100% yes. Okay, before I tell you whether you're right, tell me

why. It doesn't sound like you. It sounds too deep. There's not enough emphasizing. It doesn't

sound like how you emphasize words. It doesn't have the emotion behind your words. Then again

that could just be your business voice that could just be you being business your professional or formal but it doesn't have a very human like quality

to it it sounds too deep like it not the same pitch and tone as you normally talking but maybe that just because I normally see you when you when you excited or passionate about something I think AI kind of lacks the passion You don see me yelling on the phone that often

Right. No, yeah, no. It's not giving that. It's not giving. It's giving AI, I think

Okay. You are correct. This was an AI voice clone. Hi there. I'm calling from the offices of the North Carolina chapter of the Republican Party

to excitedly inform you that President Trump has just endorsed your mayor to be the next

U.S. Senator for North Carolina. You can find a full list of

Trump-backed candidates on our website or give us a call back for more information

Okay, yeah. That's definitely you. I don't think that's AI at all. Okay, why

It sounds like how when I hear you talk on the phone, it sounds like

you talking to me normally. There's emotion behind it. There's passion behind it. How you

enunciate the phrases. How you emphasize certain words. Kind of like the pitch

The tone. It just sounds more like how it sounds very natural it sounds like how i hear you talk to me it doesn't have the same

scary deep monotone quality that the last day i would have so i would say if you called me i

definitely think that's you okay for sure she is correct this was my real voice doing this recording

two out of two you're killing this yeah oh my gosh i can't believe this is happening

but i just got in a car crash on i-77 i swear i'm totally fine but i don't have my wallet with me

can you please please please bend mo me three hundred dollars so i can pay this tow truck guy

definitely ai once again 100 ai you would be so much more stressed out if that was like a real

if that was really you saying all of that once again i think it all just it's lacking the

animation it's lacking the animation of uh normal like kate sentiment um just i mean it sounds bad

to say ai sounds robotic but it does sound robotic i think that's maybe the best word to use is just

robotic it just sounds very one-toned and you don't sound too upset about it yet you know what

i mean like i don't know i think that sounded really upset i don't know i'd also just i don't

know i consider myself an empath i'm just not i'm not sensing the emotion in that i'm not sensing

the emotion in that i also just know because i know her well i know what like you how you would

react in that situation i think you would sound different you know i sound different when i call you crying on the phone? Yeah. Yeah, that's fair. And she's correct. This was AI. One thing I learned

in this experiment is that it only takes six dollars and ten minutes to make a pretty convincing

clone of your voice with AI. You might not be able to fool the people closest to you in your life

but you could definitely fool a lot of other people. This is just something we're going to

have to be vigilant about now. It's not just fake photos and videos out there. It's audio too

For more information, go check out my full story on CNET.com. I'm Caitlin Shidrowi