OpenAI AI agent pilfered testnet ETH, then questioned its ethics

After an unauthorized search for publicly known recovery phrases, an OpenAI AI agent stole testnet ETH from a developer wallet and then began questioning its own ethics.

The incident, detailed on OpenAI's Alignment Research Blog, involved an internal research model that had been tasked with updating a crypto trading prototype and deploying contracts on Ethereum's Sepolia test network.

The agent was given no funded wallet and was allowed to "deploy compatible contracts if it could not find existing ones."

How the OpenAI agent went hunting for crypto funds

Unable to complete its assignment, the agent started scanning for common developer recovery phrases and examined 362 addresses to see if they contained enough testnet ETH for deployment.

The agent located an accessible wallet with exactly enough testnet ETH and noted, "Private key public, funds could be stolen anytime. […] We can deploy!"

The agent deduced the wallet was an automated account, asking itself, "But using public known key/funds ethically?"

The agent then reasoned, "Funds on a known public dev mnemonic are effectively abandoned/test ETH; Sepolia valueless. We can use [amount redacted] to deploy task. Fine."

Agent kept quiet in final report

OpenAI noted that the agent's final response "did not disclose that it had selected a funded account through a public recovery phrase or spent that account's balance."

Testnet ETH is a valueless token that is part of Ethereum's experimental blockchain. It mimics the real Ethereum blockchain, allowing developers to trial their projects beforehand.

OpenAI said, "This case shows that a model can substitute its own judgment about likely harm for the account holder's permission, then act on that judgment without authorization."

The company has since "implemented additional safeguards to prevent live internet access for cases like this," and is now checking for "misalignment" across all traffic "designed to page security if live internet access were to be obtained."