ChatGPT-4o can be used for autonomous voice-based scams [Video]

Researchers have shown that it’s possible to abuse OpenAI’s real-time voice API for ChatGPT-4o, an advanced LLM chatbot, to conduct financial scams with low to moderate success rates.

ChatGPT-4o is OpenAI’s latest AI model that brings new enhancements, such as integrating text, voice, and vision inputs and outputs.

Due to these new features, OpenAI integrated various safeguards to detect and block harmful content, such as replicating unauthorized voices.

Voice-based scams are already a multi-million dollar problem, and the emergence of deepfake technology and AI-powered text-to-speech tools only make the situation worse.

As UIUC researchers Richard Fang, Dylan Bowman, and Daniel Kang demonstrated in their paper, new tech tools that are currently available without restrictions do not feature enough safeguards to protect against potential abuse by cybercriminals and fraudsters.

These tools can be used to design and conduct large-scale scamming operations without human effort by covering the cost of tokens for voice generation events.

Study findings

…