AI GPT-6 Astra mined a diamond in Minecraft overnight by directly controlling a computer

Developer Wuyang Zhou gave GPT-6 Astra one task – mine a diamond in Minecraft – then hit record and went to bed. By morning the diamond sat in the inventory, and the toolkit that made it all work went public shortly after.

The interesting part is how the agent actually plays. There's no mod, no access to game memory, no API of any kind. The model just watches the screen, moves the mouse, and presses keys like a person would.

The Codex-Minecraft-Gameplay toolkit is a Windows-only package that does exactly three things. It feeds screenshots to the agent, sends keyboard and mouse input events into the game, and optionally lets the model check its results visually.

The documentation states outright that the package includes no model, no game client, no world save, and no interface to Minecraft internal memory. Every interaction runs through the visible game interface alone.

Setup takes a handful of steps. The folder needs to open as a local Codex project on the same Windows machine running the game, then someone loads a world, keeps the Minecraft window visible, and hands the agent the repository along with the in-game goal.

Zhou also pointed out that the toolkit carries zero built-in knowledge about the game. The agent worked out every skill on its own after receiving the task, and the source code went public specifically to make future runs easier to set up.

Nobody gave the model specific instructions like "dig branch mines." Astra picked branch mining – digging long parallel tunnels – entirely on its own, working only from the stated goal.

In a separate run using the same world seed, the model landed on a cleverer trick. After dying twice from fall damage, it placed a boat to cushion the landing and made it to the bottom of the cave alive.

Comments under the post carry about as much skepticism as praise:

• The run happened on peaceful difficulty, where mobs don't spawn and hunger doesn't drain, reducing the task to a straightforward search

• Around the three-minute mark the agent walked right past a diamond ore block without noticing – a mistake a human could make too, though far less likely

• The F3 debug screen got toggled on and off repeatedly for no apparent reason

• Astra broke its crafting table with a pickaxe instead of an axe

• Torches ended up placed in spots that didn't need any lighting at all

Token usage went unreported, and Zhou publicly asked an OpenAI staffer to reset his limit so he could run longer sessions. The team's next step involves distilling the recorded actions into a local 27-billion-parameter Qwen model.

Zhou built the toolkit together with Tianyu Wei. The repository ships under the Apache 2.0 license and is available here.

The diamond challenge itself has served as a benchmark for a while now. The MineRL-based Minecraft Diamond environment builds around twelve sequential stages leading up to the diamond, requiring progress through a tech tree with more than 379 crafting recipes.

Computer control has become the main focus for GPT-6 Astra, which launched on September 3, 2026. In latency-simulated runs on the OSWorld 2.0 benchmark, the model scores 72.6% against 65.7% for the previous GPT-5.6 Sol, and finishes tasks in about 40 minutes instead of 75.