Remix.run Logo
robryk 4 days ago

Would automated tests that produce a transcript of what they've done allow perusing that transcript to substitute for manual testing?

pjc50 4 days ago | parent | next [-]

That sounds harder?

There's a lot of pedantry here trying to argue that there exists some feature which doesn't need to be "manually" tested, and I think the definition of "manual" can be pushed around a lot. Is running a program that prints "OK" a manual test or not? Is running the program and seeing that it now outputs "grue" rather than "bleen" manual? Does verifying the arithmetic against an Excel spreadsheet count?

There are programs that almost can't be manual, and programs that almost have to be manual. I remember when working on PIN pad integration we looked into getting a robot to push the buttons on the pad - for security reasons there's no way of injecting input automatically.

What really matters is getting as close to a realistic end user scenario as possible.

simonw 4 days ago | parent | prev | next [-]

No. I've fallen for that trap in the past. Something inevitably catches you out in the end.

bluGill 4 days ago | parent | prev [-]

The value of manual tests is when you "see something" that you didn't even think of.