الاختبار والقبول: متى تُعد البرمجية «منتهية» فعلاً؟

Testing does not guarantee that software is bug-free; it guarantees that known scenarios work. That distinction shortens half of every delivery conversation: nobody can say 'no bugs will appear', but you can say 'these hundred scenarios were all run and passed'. That second sentence is the acceptance criterion; the first is a wish, the second is a record.

Testing's second function is to make discussion objective. Sentences like 'the system is slow' or 'this screen is confusing' become measurable once turned into a test item: on which screen, with what data, in how many seconds. What wastes the most time in acceptance is vague sentences where everyone means something different.

The testing side of a delivery should run on a numbered, tickable catalogue. Every item has a fixed id, a precondition, a list of steps and an expected result; the result either passes or fails. The fixed id matters because saying 'that test failed again in the second wave' is only possible if the test has a stable name.

In our own products we have grown that catalogue to hundreds of items and keep it on a single tickable page. Team members tick their own sections and what remains is visible at a glance. The most unexpected benefit turned out to be psychological: instead of an undefined pile called 'to be tested', there is a list with a known size that keeps shrinking.

The catalogue is also the delivery document. In the acceptance meeting, the answer to 'what was tested' is that list rather than anyone's memory, and the remaining items are visible too. The healthiest relationship with a client comes not from hiding gaps but from reading the list of gaps together.

A developer tests expecting the thing they wrote to work; a user simply uses it without knowing what to expect. So developer testing verifies the happy path while user testing finds the unhappy ones. They do not substitute for each other: a product delivered on developer testing alone breaks in unexpected ways with its first real user.

In user testing the most valuable findings are not errors but hesitations: the moment a user pauses for a second and thinks 'what am I supposed to do now'. Those moments never become bug reports, yet they decide the product's real quality. That is why in test waves we ask not only 'did it work' but also 'was there a point where you got stuck'.

One round is not enough, because fixing the bugs produces a new version — and that version has not been tested. The right structure is in waves: test, fix the findings, run the same list again. The second wave is faster than the first because most items already pass; by the third, usually only edge cases remain.

The most common mistake between waves is sharing the test guide as a new file each time. By the second wave nobody knows which file is current. Instead we use a single address: the guide lives at the same link and is updated for every wave. For the participant there is exactly one place, and that place is always right.

Automated tests exist for the things it would be absurd for a human to re-check every time: business rules, calculations, authorisation boundaries, data transformations. Such tests are written once and run for free on every change; that is where their value lies — they protect not today, but the thing someone will unknowingly break six months from now.

Where they fall short is the interface. No automated test tells you whether a screen is usable; alignment, readability and a heading that breaks on a narrow screen are seen only by a human. Our balance: server-side automated tests for business logic and authorisation, numbered human testing for the interface. Investing fully in both at once is needlessly expensive on most projects.

There is an incident behind putting 'with an unauthorised account, try something it should not reach' on the test list. During a short post-release check we noticed that a call made with an ordinary user account could return data belonging to an organisation it did not belong to. Screens opened, the code raised nothing, all tests passed — because no test was trying what the system should refuse.

The general rule from this: some of your tests must be negative scenarios. Not only 'does the right user see the right thing' but also 'is the wrong user unable to see the wrong thing'. In multi-tenant systems this is not an optional check but a mandatory item; one tenant's data appearing to another is not a fixable bug but a loss of trust.

A good bug report contains three things: what I did, what I expected, what happened. With those three written down the developer has nothing left to ask; without them the report turns into a thread and the fix stretches over days. In test waves, the highest-return move is giving participants that three-line template up front.

Two additions help: a screenshot and the record's id. A screenshot explains what 'it looks confusing' means at a glance; the id shows which data produced the fault. When the same screen works on one record and breaks on another, the difference is in the data itself — and you find it only through the id.

Add a note on reproducibility too: does it happen every time, or was it seen once? Even 'it happened once and I could not reproduce it' is valuable, because it tells the developer this may be a timing or data-state issue. Leaving non-reproducible faults out of the record makes the hardest bugs invisible.

The soundest measure for prioritising findings is the question 'can we ship with this as it is'. Nothing that causes data loss, a wrong calculation or unauthorised access can ship — those are blockers without debate. Findings that slow work down but have a workaround can ship; appearance and comfort findings can wait for the next release.

Those three levels look simple, and simplicity is exactly what ends the argument. Five-level severity scales end in practice with everyone marking their own item 'high'. Whoever assigns the level, the owner of the decision should be the person who knows the business — a developer knows how hard a fix is, not how important it is.

A fixed bug returning is called a regression, and the cause is usually this: the bug was fixed but no test was written to catch it. Months later another change touches the same code and nobody notices. Hence our rule: if a bug has been seen twice, an automated test is written along with the fix.

The second cause is fixing the symptom. Patching the visible result without reaching the root brings the bug back from somewhere else. The practical way to tell is this: can the person who made the fix answer 'why was this happening' in one sentence? If not, the symptom has probably just been covered up.

A good acceptance criterion is concrete enough to leave no room for debate, and it is written in business language. Not 'the report screen will be fast' but 'the report opens in five seconds with a month of data'. Not 'there will be user management' but 'an administrator can add a user, change their role and deactivate them'. The person writing the criterion should be the one who knows the business, not the developer.

The second rule is to tie acceptance to a list rather than a date. Acceptance tied to a date gets signed whatever the state on the day, and the gaps drift into 'we'll sort it later'; acceptance tied to a list shows plainly what is finished and what is outstanding. In our deliveries we put those two lists side by side: passed and pending.

The thing that earns the most trust in a delivery is stating plainly what is not there. On our own product pages we mark unbuilt features as 'soon'; that costs something in a sales conversation in the short term, but the price of doing the opposite is far higher later. A feature assumed to exist, discovered missing after delivery, makes the whole project debatable.

In practice this means splitting scope into three boxes: in this delivery, not in this delivery but on the roadmap, and never going to be built. The third box is usually skipped and causes the most misunderstanding. Saying that something was deliberately left out is better than letting people think it was forgotten.

The closer test data is to reality, the more useful the test. Testing with three hand-typed sample records shows nothing about the length of lists, the crowding of search results, how long names break the layout, or whether paging works at all. So a copy of real data is brought into the test environment — only then do you see how the system behaves in a crowd.

But when real data is copied, personal information must be masked: names, phone numbers, email addresses and identity details should not sit in a test environment in their real form. Test environments are usually less protected than production; data leaking from one carries the same liability as a leak from the other. The masking script should be an inseparable part of the copying script.

Then there is edge data: the longest name, the largest file, the most crowded list, a back-dated record, a record belonging to a deleted user. These rarely occur in real data yet always turn up in real life. Adding a few edge items to the test catalogue prevents hearing 'we thought nobody would try that' after launch.

Delivery ends not when the software works but when the client can run it. The test is simple: whoever will update the content — can that person change a piece of text, add a user, unpublish a record, with no help at all? If the answer is no, the product is finished but the delivery is not.

The second leg of handover is having how the system works written down: which environment is where, how a release is made, where backups come from, who does what if something breaks. That document need not be long; one page of correct information beats thirty pages of boilerplate. We keep these notes in the project's own memory files, because the person asking a year later is usually us.

Software is not done when its list runs out; it is done when its list becomes visible. A numbered test catalogue, checks repeated in waves, a security item that includes negative scenarios, acceptance criteria written in business language and an honestly marked list of gaps — these are what turn a delivery from an argument into a record.

This series of three articles covers three steps of delivery: going live, data migration, and testing and acceptance. If you have a project waiting to be delivered, or if your current process gets stuck in the same place every time, we can start by talking about which step you are on.