A Fraudster’s Paradise – O’Reilly

Darkish internet discussion board posts talked about the phrase “AI agent” way more within the second half of 2025 than within the first half. Might this imply that fraudsters are charmed by the AI hype? Or is AI really a recreation changer for cybercrime? AI-related discussions—evident each in what “the dangerous guys” are saying and in what fraud-fighters are exploring—are in every single place. In actual fact, a Visa PERC evaluation evaluating knowledge from June to November 2025 to the six months earlier than that discovered an improve of greater than 450% in darkish internet agentic AI-related posts.

Documented monetary losses from deepfake-enabled fraud exceeded $200 million in Q1 2025 alone. And Visa reported a 25% improve within the second half of the yr in malicious bot-initiated transactions. There’s AI-generated rip-off and fraud content material in every single place you look, and it doesn’t all the time give Eric Clapton six fingers to make it apparent that one thing’s off:

None of that is taking place in a vacuum. On-line fraudsters have embraced GenAI with open arms in a approach that has modified the form of the web fraud panorama. On the identical time, odd shoppers are leaning into the form of dishonest that GenAI makes simple.

How can the monetary and digital ecosystem combat again towards this new wave of AI-powered fraud? Studying from previous expertise, collaboration and information sharing between fraud-fighting professionals is essential to our collective capability to brace for impression. This quick weblog put up is a name for harnessing the ability of AI for enhancing the know-how of digital defenders, as a option to strengthen our general protection.

Keep in mind COVID?

AI isn’t the primary time that the principles of cybercrime have modified dramatically (and it gained’t be the final). For instance, again in 2020, in the course of the first COVID-19 lockdowns, work-from-home schemes exploded, together with first-party fraud, phishing scams, and extra. On the time, the neighborhood responded successfully. Groups of fraud-fighting consultants joined forces to satisfy nearly, be taught the brand new terrain and finally write a research information that might empower organizations to protect communities from a surge of digital fraud. Our ebook Sensible Fraud Prevention was the results of that collaboration, and it rapidly grew to become a useful coaching useful resource to many groups within the ecosystem of their combat towards on-line monetary crime.

As we speak, a brand new wave of fraud is rising, powered by AI and particularly GenAI. Whereas some AI-powered initiatives are right here just for a short time, others have gotten really highly effective and harmful instruments in nefarious fingers. It’s solely pure that the skilled neighborhood will as soon as once more regroup to type a playbook towards these tendencies.

As we interview consultants in numerous fields for our subsequent ebook, The Fraud Fighter’s AI Playbook (with coauthor Chen Zamir, now in early launch on O’Reilly), we’re constructing an image of the ways in which GenAI is altering not simply the methods fraudsters function on-line but additionally the form of the web fraud panorama itself. We’re seeing, too, how very important it’s that fraud fighters themselves put money into exploring and utilizing this expertise to spice up their success, technique, and inside repute in their very own corporations.

Extra fraudsters, doing extra hurt

There are extra on-line fraudsters immediately, finishing up extra fraud assaults, than ever earlier than. Not all of this may be blamed on GenAI. Somewhat, GenAI entered right into a fraud world that, looking back, was poised to leverage it for crooked growth.

The COVID-19 pandemic drew many new fraudsters on-line, via three primary tracks:

• Digital transformation + time to spare. Retailers, banks, and different organizations needed to cram years of digitisation into weeks or months. It was inevitable that this is able to lead to some vulnerabilities in processes, insurance policies, or methods. Folks caught at house with no work had been tempted to make some cash exploiting these weaknesses and realized quick.
• COVID aid packages. Applications put in place to tide of us over the difficulties of the pandemic didn’t all the time embrace checks to confirm identities or claims. Those that realized tips on how to make fraudulent submissions, typically even by creating faux or artificial identities or companies, discovered how simple and the way profitable that may be.
• Rip-off compounds. Human trafficking rings pivoted from in-person exploitation to forcing individuals on-line to hold out phishing assaults and scams of every kind.

None of those tendencies has disappeared, and the rip-off compounds specifically have expanded massively because the finish of the pandemic. This was the world that GenAI was born into.

With GenAI, way more fraud assaults are potential, at a degree of personalization that might have been inconceivable at scale with out the expertise. Widespread makes use of of GenAI within the wild embrace:

• Phishing campaigns, personalised by utilizing open supply details about the goal (sourced by way of GenAI), and the suitable language and cultural touchstones for every sufferer (made simple by placing conversations via GenAI). Extra typically, the impression is seen in scams of every kind, from romance scams to funding scams to kidnapping, catfishing, blackmail scams and extra—equally personalised. 

• Malware and bot creation to hold out assaults, steal info, arrange faux accounts, and extra. Fraudsters now not want a lot technical capability to create malicious packages or automation, enabling extra fraudsters to amp up their attain. 
• Deepfakes. Whether or not for clickbait technology or for monetary scams, deepfakes permit attackers to move id validation checks. In stunning circumstances, deepfakes had been even used to faux the trail to a job (and the wage and entry to knowledge that comes with it). Broadly, AI simplifies the method of making faux or artificial identities utilizing open supply info, paired with stolen private credentials.
• Pretend web sites, faux apps, faux promoting. Manipulated content material will be utilized to influence shoppers to buy nonexistent or low-quality merchandise, however it might additionally goal digital promoting. Main manufacturers are shedding thousands and thousands of {dollars} to AI schemes that generate views, impressions, or clicks for undeserved advert income. A research of faux cellular apps, led by Gilit’s staff at DoubleVerify, discovered that faux iOS grew to become 3 times extra frequent (accompanied by six instances extra faux Android apps) in 2025 in comparison with earlier years, a development amplified by AI.
• Fast evolution in sophistication and evasion of bot schemes. Agentic AI is getting higher daily. Which means that operators of bot networks are smirking at CAPTCHA challenges (“resolve this puzzle to show you’re not a bot”). Fraud assaults of 2023 had some rookie errors, like bot networks that attempted to move as human beings watching TV content material besides that they had been coming in with gadget settings of fridges. 2026 attackers will now not fail at fundamental deception as a result of they’ve AI chatbots (FraudGPT, WormGPT, and many others.) to information them on their approach.

The excellent news—if there’s any actual excellent news—is that a minimum of these aren’t new varieties of assaults. They’re acquainted assaults, carried out extra convincingly, at far higher scale.

GenAI is a aspect hustler’s finest pal

It’s not simply devoted fraudsters utilizing GenAI to broaden their attain. Bizarre individuals use it to degree up their dishonest recreation too.

Refund fraud has grow to be simple at a really convincing degree because of GenAI. Many retailers ask for photographic proof that an merchandise has arrived damaged or broken, and with GenAI, that’s one thing that may be faked in seconds. Because the picture is created for a goal from scratch, there’s no option to discover an unique on-line as proof that it’s a cheat.

Some individuals have gotten much more artistic, utilizing the identical form of trick as a part of an insurance coverage declare. Others use GenAI to whip up faux receipts, which they will declare again from their firm.

It’s vital to notice that, as with the skilled use circumstances, it’s not that these cheats are new sorts of assaults. What’s new is the convenience, scale, and class with which they are often carried out.

The bottom shifting below our ft

Once we wrote Sensible Fraud Prevention, we included a dialogue of issues like phishing, victim-assisted fraud, refund fraud, and so forth. The concentrate on the ebook, although, was on the ways in which fraudsters money out their schemes. Comply with the cash, and discover the fraudster.

Now, solely just a little greater than three years after ChatGPT burst into all of our lives, that emphasis has shifted. Conventional third-party fraud, the sort you get when a fraudster makes use of your bank card on-line, isn’t even within the prime three fraud issues. TransUnion studies that the most enterprise loss in 2025 got here from rip-off/approved fraud (24%), adopted carefully by artificial id fraud (20%) and account takeover (20%). That’s the GenAI impression.

There’s a big price ticket hooked up to numbers like that. The identical report famous that “corporations worldwide misplaced 7.7% of their annual income on common resulting from fraud over the previous yr.” Within the US, it was 9.8%.

There’s additionally a worrying impression on belief. When deepfakes are frequent and convincing, who can ever imagine their eyes? Clients don’t know which internet sites are actual, which messages are genuine, or which adverts or gives will be trusted. Companies don’t know which claims are respectable or how finest to remain forward of the verification challenges they now face. Marketplaces battle to guard patrons from dishonest sellers, and sellers from dishonest patrons, and everybody from exploitation by malicious actors.

Ha! Fraud fighters have GenAI, too

The ray of hope in our analysis is that fraud fighters have GenAI too, and groups are already experimenting in a wide range of methods. Some are taking a look at how they will use brokers to broaden their open supply analysis to make their selections quicker and extra correct. Others are engaged on tips on how to craft prompts to assist analyze knowledge or work out tendencies that can be utilized to determine and cease fraud. Nonetheless others are leveraging GenAI to investigate documentation, to select fakes or alterations. And so forth.

It’s additionally encouraging to see how groups are utilizing GenAI to broaden their attain internally inside an organization. In some methods, it’s virtually like fraud departments are getting the assistant they’d all the time wished to do the duties they’d all the time meant to get to—like pulling the info and placing it collectively for a biweekly replace to related stakeholders or creating detailed materials with useful illustrations or graphs for shows to different departments.

It’s inevitable that when a completely new expertise comes alongside, the fraudsters can have an higher hand initially. They aren’t hampered by concerns like regulatory issues or authorized necessities, they usually don’t care about issues like accountability, duty, or shopper belief. It’s just about of their job description to disregard these issues, the truth is.

The fraud-fighting business has been sensibly cautious about understanding tips on how to determine and make use of GenAI, however it’s clear that they’re not standing nonetheless. The groups who do the most effective with this evolving problem would be the ones who work carefully and persistently with departments throughout their firm to adapt rapidly to the enterprise’s wants—and tips on how to meet them.