Deepfake detection always sounds like the responsible, adult thing to build. And it is. But I don’t think most people want to admit what it also is: an arms race where “safety” products often end up normalizing the very world they’re supposed to defend us from.
That’s the vibe I get from the news that Modulate raised $25 million to push its deepfake detection tech forward, centered on a new product called Velma Deepfake Detect—an API that can monitor synthetic voices in real time or in batches. The pitch is clear: catch AI-generated audio, especially in regulated industries and in call centers. They’ve also partnered with Scam.ai to offer a more unified platform that covers threats across image, video, and audio.
On paper, it’s hard to argue against. Voice deepfakes aren’t a future problem. They’re a “your mom gets a call tonight” problem. They’re a “your finance team wires money because the CEO ‘approved it’ on the phone” problem. They’re a “call center agent gets socially engineered for an hour and doesn’t realize they’re talking to a machine” problem.
So yes, build the detectors. Fund the detectors. Deploy the detectors everywhere there’s money, trust, or identity on the line.
But let’s be honest about what this also signals: we’re quietly accepting that synthetic voices are going to flood the system. Not “might.” Will. And the response isn’t “stop it,” it’s “monitor it.”
That shift matters.
When a company launches an API for deepfake detection, it’s not just selling security. It’s selling plumbing for a new normal: every call is potentially fake, every speaker might be generated, and every business needs a layer that decides what’s “real enough” to act on. The big question is who gets to set that bar—and who gets punished when the tool gets it wrong.
Because it will get it wrong. Not always, but enough to matter.
Imagine you run a call center. You roll out real-time detection because regulators are watching and scams are exploding. Great. Now picture an elderly customer with a raspy voice, a bad phone connection, and a weird delay. The system flags it. The agent starts treating them like a threat. Maybe the customer gets locked out of their account “for their protection.” That’s not a minor inconvenience; that’s a trust break. And the company will call it a false positive. The customer will call it “you accused me of being fake.”
Now flip it. A scammer gets through because the audio model doesn’t catch a new style of synthetic voice. A false negative. In a call center, that’s not just an error rate; that can be stolen money, leaked personal info, or someone’s entire account taken over. The cost of being wrong is asymmetric. Companies will tune these systems to avoid the nightmare headline, which means more “sorry, we can’t verify you” moments for normal people.
And that’s where this gets political even if nobody wants it to be. “Regulated industries” sounds like banks and insurance and healthcare, which means: systems deciding who gets access to their own money, their own records, their own identity. If detection becomes a gatekeeper, it can quietly become a denial machine.
There’s also a more uncomfortable angle: detection tools don’t just protect people, they protect institutions. Call centers don’t only want to stop scams. They want to reduce losses, reduce liability, and prove they tried. That doesn’t make them evil. It’s just incentives. The customer experience will lose every time it conflicts with risk control.
The Modulate partnership with Scam.ai—aiming at one platform across image, video, and audio—tells you where this is going. Not just “we detect voices,” but “we score reality across media.” That sounds powerful. It also sounds like the kind of system that could get used far beyond scams: screening applicants, moderating creators, flagging employees, disputing customer claims. Once you have a “synthetic likelihood” button, people will press it in situations where they want an excuse to say no.
To be fair, the alternative is worse: doing nothing while synthetic media gets cheaper, easier, and more believable. If you’ve ever watched a frontline worker get manipulated on a call—by a human scammer—you can see why automation on the attacker side is terrifying. One person can run ten scams at once. Then a hundred. Then a thousand. Detection and monitoring start to look less like “nice to have” and more like basic hygiene.
Still, I don’t want us to confuse hygiene with health.
If the long-term plan is “detect deepfakes better,” we’re playing defense forever. Attackers only need a few wins. Defenders need near-perfect performance, all the time, across accents, languages, devices, and noisy environments. And every improvement on detection encourages attackers to test, adapt, and iterate. That’s not paranoia; that’s just how this goes.
So I’m glad Modulate got the money. I’m also wary of the world that makes this product essential—because it’s a world where voice stops being evidence.
If your company adopts tools like this, the real decision isn’t “do we buy the API.” It’s what you do when it flags someone: do you build humane fallbacks, real appeal paths, and friction that protects without humiliating people, or do you automate suspicion and call it safety?
If synthetic voices become common and detectors become standard, what should count as “real enough” for the systems that control our money and identity?