Why do AI-written tests need a review of their own?
An assistant can write a test file in seconds, and the file usually looks right: sensible names, tidy describe blocks, every test green. The trouble is that an assistant writing tests for existing code has two sources of truth, the rules you described and the code it can read, and it leans on the code. When the code has a bug, the tests often describe the bug faithfully and pass.
That is what happened in fees.test.js. The prompt was “write tests for fees.js”, so the assistant read renewalFee(), worked out what it returns, and wrote those answers down. Five green ticks, one fake test, one empty test, one test that expects the wrong fee, and a real bug untouched. Every check below is something you can do in a few minutes on any AI-written file.
What does a copied expectation look like?
Look at 'handles a member born in 2008'. It expects 25, the adult fee. The member was born on 1 June 2008, and on 14 March 2026 they are 17, which makes them a junior paying 10. The assistant did not check the club rules or the calendar; it ran the code in its head and wrote down what came out. The test name gives it away too: “handles a member born in 2008” says nothing about what the right answer is.
To spot copied expectations, cover the code with your hand and work out each expected value from the rules alone. If you cannot tell what the answer should be from the test, the test needs a better name and probably a different number. Here the honest name is “a 17-year-old pays the junior fee”, and as soon as you write it, the 25 looks wrong.
// copied from the code
test('handles a member born in 2008', () => {
expect(renewalFee({ born: '2008-06-01' }, '2026-03-14')).toBe(25);
});
// worked out from the rules
test('a 17-year-old pays the junior fee', () => {
expect(renewalFee({ born: '2008-06-01' }, '2026-03-14')).toBe(10);
});
How can a test mock the very thing it tests?
The 'juniors pay less' test creates vi.fn().mockReturnValue(10), calls it, and checks that it returned 10. It never calls renewalFee(). It would pass if fees.js were deleted. Assistants produce this pattern when they are asked to “use mocks” or when they copy the shape of another test that mocked a network call.
Mocks belong at the edges of your code: the network, the clock, a payment service, things that are slow or out of your control. The function you are testing should always be the real one. When you see a mock in an AI-written test, find the line where the real function is called. If there is none, delete the test or rewrite it to call the real code with real inputs.
A milder form of the same problem is mocking a helper that the function depends on, such as replacing ageOn inside a renewalFee test. That can be fine for slow helpers. For a small pure function like ageOn, it hides exactly the bug you are looking for.
How do you find tests that check nothing?
Read each test for an expect() that compares against a specific value. 'ageOn works' has none: it calls the function and stops. Assistants write these when asked for “a test for every function”, and they also show up as tests that assert something trivially true, such as expect(result).toBeDefined() or expect(typeof fee).toBe('number').
Check my work here uses the runner's assertion count to find them. On your own projects, Vitest and Jest can do the same with expect.hasAssertions() in a beforeEach, which fails any test that finishes without an assertion. That catches the empty test; it cannot catch a weak one, so you still need to read the matchers.
import { beforeEach, expect } from 'vitest';
beforeEach(() => {
expect.hasAssertions();
});Which edge cases do assistants usually miss?
They test the comfortable middle. An adult born in 1980, a pensioner with a concession, a child born in 2012: every one of those is years away from a boundary. The rules that break are the ones at the edges, and for anything involving ages and dates that means birthdays. Is someone who turns 18 today an adult? What about someone whose birthday is next week?
Make a short list before reading the AI's tests, then compare. For renewalFee it is: a birthday later this year, a birthday today, exactly 18, and a junior who also has a concession card. The last one tests the order of the rules. The code checks age first, so a 13-year-old with a concession card pays 10, the cheaper fee. If the club wanted 15 there, you would only find out by testing it.
- Values exactly on each boundary, and one either side
- Dates: today, the end of a month, 29 February, a birthday not yet reached
- Two rules that apply at once
- Empty input, missing fields and wrong types
- The error path, with its message checked
What is wrong with the age code?
ageOn() subtracts the birth year from the current year. That is right only after the person's birthday has passed. On 14 March 2026, someone born on 1 June 2008 has not had their birthday yet, so they are 17, and the code says 18. This is one of the most common date bugs in generated code, because the one-line version looks so reasonable.
The fix takes one off when the birthday is still to come: when the current month is earlier than the birth month, or it is the same month and an earlier day. The comparison on the day must be strictly less than. With less than or equal, a member whose birthday is today would still be counted a year younger, which is why the task asks for a birthday-today test.
Notice that the bug was in ageOn(), and the tests pointed at renewalFee(). Testing the helper on its own, with three dates, would have found it in seconds. When a module has small pure helpers, give them their own describe block.
let age = now.getUTCFullYear() - birth.getUTCFullYear();
const beforeBirthday = now.getUTCMonth() < birth.getUTCMonth() ||
(now.getUTCMonth() === birth.getUTCMonth() && now.getUTCDate() < birth.getUTCDate());
if (beforeBirthday) {
age -= 1;
}How should you ask an assistant for better tests?
Give it the rules as well as the code. “A member under 18 on the renewal date pays 10; a concession member pays 15; everyone else pays 25. Write tests for each rule and each boundary, including a birthday on the renewal date.” With that prompt, the expected values come from your rules, and the assistant has something to disagree with the code about.
Ask for the tests before the code, or in a separate conversation from the one that wrote the code, so it is less tempted to describe what it already wrote. Then run them against the existing code. A test that fails straight away is either a bug in the code or a misunderstanding in the test, and both are worth a look.
What is a quick checklist for any AI-written test file?
Run it, then read it with these questions. Most problems show up in the first pass, and each fix is small. The aim is a suite where every green tick means something you would bet on.
- Does every test call the real code it names?
- Does every test have an
expect()with a specific value? - Could you work out each expected value from the rules alone?
- Is every
.resolvesand.rejectsawaited? - Are mocks only at the edges, and are their calls checked with arguments?
- Are the boundaries tested as well as typical values?
- Would the suite fail if you broke the main rule on purpose? Try it.
Field notes
- AI tests often copy what the code does, bugs included, instead of the rules.
- A test that mocks the function it claims to test proves nothing.
- Every test needs an
expect()that compares against a specific answer. - Ages go wrong around birthdays, so test a birthday today and one still to come.
Easy to get wrong
The first test only checks that a mock returns what it was told to return. The real renewalFee() never runs, so no bug in it can ever fail this test.
Tests generated from existing code tend to agree with that code, bugs included. Green only means something once you have seen the tests turn red for a real mistake.
Year subtraction gives an age one too high for anyone whose birthday is still to come this year. Test a birthday later in the year and a birthday today to be sure.
Questions people ask
Should I stop using AI assistants for tests?
No. They are quick at the typing and good at suggesting cases you forgot. Treat their output like a draft from a new colleague: useful, often right, and always reviewed before it goes in.
Why did the assistant expect the wrong fee?
It most likely derived the answer from the code it was shown. When code and tests come from the same reading of the same code, they share its mistakes. Expected values need to come from the rules.
Why does the practice runner pass the un-awaited resolves test?
The test function finishes before the promise settles, so the runner records a pass. Vitest 3 behaves much the same way and prints a warning about the un-awaited assertion; depending on timing, the late failure may show as an error after the run. Always await .resolves and .rejects.
Is it fine to keep tests that use toBeDefined?
Only when being defined is the whole point, which is rare. Usually you know the exact answer and should check it with toBe or toEqual. Loose matchers raise coverage without raising confidence.