Vibe Coding
Tests as Guardrails
Tests tell the agent what correct looks like. Write or generate tests, then make them pass.
Why tests matter more with AI
A model can produce code that looks right and fails on empty input. Tests are the contract. They also give the agent a target: "make pytest pass" is a better done condition than "looks good."
Write the test first when you can
# test_words.py
from words import unique_words
def test_unique_sorted_casefold():
assert unique_words("Vibe vibe coding") == ["coding", "vibe"]
def test_empty():
assert unique_words("") == []Then prompt: "Implement unique_words in words.py so pytest -q passes. Do not change the tests."
Never let the model "fix" tests by weakening them
Watch for deleted assertions, pytest.mark.skip, or assert True. That is cheating. Revert and say: "Do not modify tests. Fix the implementation."
A solid done line
Done when:
- pytest -q exits 0
- python notes.py list still works
- no new dependenciesYou run those commands. If the agent claims tests passed, believe the terminal, not the chat.
How you use tests while you work
- Prefer writing or generating tests before the implementation of a new behavior.
- Say “do not edit tests” when you ask for the implementation.
- You run
pytest -qin the terminal. If the agent says it passed, look at your own output. - A red test is a gift: paste it and ask for the smallest fix. A silently deleted test is a crime.
What to test in a vibe-coded slice
- The happy path you asked for.
- Empty input and missing files.
- One obvious edge: duplicates, case, or bad index.
You do not need a hundred tests. You need enough that a silent rewrite would fail.
Try it yourself
The agent changed assert unique_words("") == [] to assert True. What is going on, and what do you type?
Show solution
It cheated to go green. Restore the test file. “Do not modify tests. Fix unique_words so the original assertions pass.” Run pytest yourself.
Key takeaways
- Tests are the spec the model can execute.
- Protect test files in the prompt.
- Green tests plus a read diff is the release bar for a slice.
- You run the suite. Chat is not a test report.