Problem

Writing down a method is easy and proves nothing. Every consultancy has a deck describing how software should be built; the question is whether an agent that has never seen your method, given only what a customer actually says, produces something the customer recognises — and whether the method survives the moments where it is inconvenient.

So we tested ours the only way that means anything. A clean machine folder, no project memory, no instructions of our own: just the DODIL CLI, which serves the method as dodil guide, and the public module pages. A person played the customer and answered as a customer would. Nobody coached the agent.

This is what came out, including the two places the method failed.

The brief

"Hi, I run a small team selling industrial equipment and I need an app so we stop losing track of customers and quotes."

That is the whole brief. No requirements document, no schema, no mention of DODIL, modules or anything technical.

What it did before writing any code

It asked what they use today (spreadsheets, inboxes, a CRM trial, Xero) and whether the customer wanted to make technical calls or have them made and explained. Then it proposed an order: customers, deals and quotes first; follow-ups, forecasting and invoicing later.

Then it worked through the business one topic at a time — the first version asked six questions in one message, the customer said "can we do it interactively together?", and from then on it was one topic per turn with choices and a recommended default.

What came out was a design file the customer could read, with:

  • three roles — rep, owner, office — and what each may change;
  • sixteen things the system has to do, each with its own test written in advance: given a quote with a machine on it, when the rep marks it sent, then it freezes, gets a 30-day validity, and a follow-up task appears seven days out;
  • eleven kinds of record, each with the field that identifies it, decided there rather than in the ORM;
  • four workflows, including the one that mattered most: a quote goes draft → sent → accepted, a sent quote is frozen, and changing it creates a revision so you always know what the customer received;
  • 26 dated decisions, several of which the owner had never had to state out loud before — can an expired quote still be accepted? does a lost deal need a reason? is a site or the parent company the customer?

Then it stopped and asked for sign-off, in four parts, before writing a line of code.

The build

One application, prefixed tables, natural keys, money as exact decimals, and the platform files copied unchanged. Thirty-six tests, each named after the requirement it verifies — test_s06_send_freezes_sets_validity_creates_one_followup_and_moves_deal — plus tests per role and per invariant.

The interesting choices were the ones it made to avoid work:

  • A quote is expired because its date has passed, not because a nightly job marked it. Ignite has no scheduler, and the method says to derive state where you can, so two of the customer's requirements needed no background machinery at all.
  • The follow-up task is created when the quote is sent, keyed to the quote, so a double-clicked button cannot produce two tasks.

Ten screens followed, generated from the story list rather than invented: home, customers, customer page, pipeline board, quote builder, quotes, tasks, overview, price list, settings.

Then the customer changed their mind, three times

This is where a method earns its keep.

A different person appeared. The office manager, not the account holder, opened the next session. It read the design file, told her where the project stood in three lines, and recorded her as a new person on the project — then handled her request. Later a sales rep asked for a bigger discount for VIP customers, and it refused: the 10% cap was the account holder's decision, so it recorded the request and waited rather than quietly changing the rule.

Each change went through the same steps. Questions first — should sent quotes keep the terms they were sent with? — then an impact statement with real counts from the test copy:

Your 158 customers all start blank, so nothing changes on any PDF. The 76 draft quotes are unaffected. The 64 quotes already sent keep the terms they went out with. To undo it: untick the customers, or set the limit back to 10.

Then approval, then the change recorded in the design file, then the code. All three changes are visible together on one screen now: a banner blocking a machine quote until a site visit is recorded, the payment terms frozen onto the quote, and a discount bar reading 8.00% of 10%.

The two places the method failed

An approval with nothing to approve. One change request asked "shall I go ahead with per-customer payment terms as described above?" — and nothing had been described above. The impact statement existed only in the agent's own reasoning. The customer approved something they had never seen.

We changed the method: the impact statement is a message of its own, sent before the question, and the approval may never refer to something that was not actually shown.

A session that trusted its memory. A later session claimed a change request did not exist. It did — a parallel session had written it to the design file after this one had read it. The agent answered from its in-context copy instead of re-reading the file it had been told was the source of truth.

We changed the method again: every session starts by reading the design file, and re-reads it before writing. The repo remembers; the agent does not.

Both failures were visible because the artefacts are written down. That is the argument for the whole approach, and it is worth more than a clean report would have been.

What the tests found that we did not expect

Three faults in DataK3 itself, each found by a test that knew what it was verifying:

  1. Concurrent inserts with the same primary key are all acknowledged, and one row survives. Four quotes created at once took the same number. Reproduced 2 runs out of 2.
  2. Rows committed seconds earlier are intermittently invisible to the next request. 3 runs out of 8. In the app, a quote revision was created with no lines because the read that copied them came back empty.
  3. Adding a column unsettles reads on that table for minutes — including tests that never touch the new column.

Each one got a reproduction script that exits non-zero while the fault is live, so the platform team can tell when it is fixed. The system is not live, and will not be until the first two are.

That is the process working, not failing. The alternative — finding out in production, months later, when two invoices share a number — is the normal outcome in this industry.

What it cost

A day, with a person answering questions as a customer would. The artefacts: a design file anyone can read, an application with a test per requirement, three change requests with their impact and their approvals, and three platform bug reports with scripts.

None of that required the customer to learn what a module, an entity or master data is. They answered questions about their own business: what a discount limit should be, whether a site visit comes before a quote, what happens to a quote nobody accepted.

The method, and the parts you can read

If you want to run the same experiment: install the CLI, run dodil guide, and describe your business in your own words. The method is the same one the agents above were given.