Day 5 with Claude: Pitfall!

Claude did a surprisingly good job with Pong and Space Invaders? How does it do with a deeper cut, like Pitfall!? Answer: it isn't pretty -- read on for the grisly details

Claude nailed Pong and Space Invaders as a single page HTML app in one shot. Both of those are popular games with many implementations in the training data. How would it do with a deeper cut, such as the 1982 classic, Pitfall!?

tl;dr — it could not get it working.

• The first attempt would not start.

• The second attempt produced garbage code

• The third attempt glitched out entirely.

One limit to vibecoding for n00bs is that debugging is really hard. Since you didn’t set anything up yourself, you don’t know why things don’t work, or even where to look. Particularly as apps get bigger, and need to be broken up into a file structure with dependencies, things get dicey. At least with platforms like Replit, you can rely on them to handle that part.

Here was the first attempt — it looked promising, but the “Start Game” button didn’t work. I didn’t know how or how to fix it — looking through the code it had all kinds of promising looking elements (like “scorpions” and “crocodiles”) that fit Pitfall!, so was it a simple bug or was there something more profoundly wrong?

I told Claude it did not work and asked it to fix it. The next swing at bat messed up the entire codebase. This is why having repos and versioning is important.

I asked it to start again, and simplifying the prompt to create a 3 level game. Once more, failed to start.

Finally, I tried one more simplification, and asked for a game where Harry needs to cross a pond by jumping on three alligators while their mouths are closed to win.

It got the name right, but there was no game and it instantly reached the win state.

So Claude can one-shot simple popular games that are common in the training data, but struggles with titles that aren’t as well documented, and don’t have many code bases lying around. This suggests LLMs may do better for isolated mechanics, than actual games if those games are obscure and outside the training set.