Don't expect cybersecurity “magic” from GTP-4o, experts warn

OpenAI CEO Sam Altman unveiled GPT-4o earlier this month, enthusing the tool's new features, calling them “magic to me.”

The company's head of developer experience, Roman Huet, said the free generative artificial intelligence tool “can infer voice, image and text in real time.” Compared to the company's previous GTP-4 model, which debuted in March 2023 and accepted text and image inputs and only output text, he said the new model is “a step toward a much more natural human-computer interaction.”

Cybersecurity expert Jeff Williams said despite the new capabilities, the model isn't expected to fundamentally change how next-generation AI tools aid attackers or defenders.

“Attackers and defenders are already imperfect. What we lack is visibility into the techniques and processes to make better decisions,” Williams, CTO of Contrast Security, told Information Security Media Group. “GPT-4o has the exact same problem, which is why it hallucinates non-existent vulnerabilities and attacks and ignores real ones.”

The jury is still out on whether such hallucinations could undermine users' trust in GPT-4o (see: Should we just accept the lies we get from AI chatbots?).

“Don't get me wrong, I think GPT-4o is great for tasks where you don't need a high degree of confidence in the results,” he said, “but cybersecurity requires a lot more confidence.”

GPT-4o's new features, such as its ability to perform multiple tasks simultaneously, could still give attackers some productivity benefits, said Daniel Kang, a machine learning researcher who has published several papers on the cybersecurity risks posed by GPT-4. These “multimodal” capabilities could be a boon for attackers who want to create realistic deepfakes that combine audio and video, he said.

While the ability to clone voices is one of GPT-4o's new features, other generations of AI models already offer this capability, which experts say could be used to commit fraud by falsely identifying someone else, such as to circumvent bank identity checks. George Apostolopoulos, founding engineer at supply chain security firm Endor Labs, said such capabilities could also be used to create disinformation and attempt blackmail (see: Top cyber extortion measures to fight virtual kidnappers).

While the security of the new AI model remains an open question, OpenAI says it has added numerous security and privacy safeguards to GPT-4o compared to previous models, including minimizing the amount of data it collects, anonymizing that data more effectively, using stronger encryption protocols, and being more transparent about how collected data is used and shared.

Kang said users still don't know what data was used to train GPT-4o, and there's no way to opt out of using large language models developed with specific training datasets. He further said users have no way of knowing exactly how the models work or whether they could be subverted. Because the tool is free, it's expected that malicious hackers and nation-state groups alike will be looking for ways to manipulate or disable it.

For CISOs, GPT-4o doesn't change the need to protect their enterprises with the right policies, procedures, and technology. This includes tightly controlling how (or whether) employees access Gen AI in their jobs, ensuring that its use complies with established security policies, and using strong contracts with suppliers to manage third-party risk, said Pranava Adduri, CEO of Bedrock Security.

“This is essentially what the cloud world went through with the shared responsibility model between the cloud infrastructure provider and the users running apps on that cloud,” Adduri told ISMG. “Here we're seeing a shared responsibility model for AI between the LLM provider and the enterprise (and its users) leveraging new applications and usage of LLM software.”

Experts also recommend never trusting publicly accessible AI models to keep them secure or private. To do this, CISOs should apply time-honored data protection principles, such as protecting sensitive, confidential, or regulated data, knowing where it flows and is stored, and applying data loss prevention policies and safeguards, Adduri said. This applies both to commercial tools that companies may develop based on other AI models or LLMs, and to employees who use tools like GPT-4o for productivity gains.