> Tests are a description

As is a spec. "Description" is literally found in the dictionary definition. Which stands to reason as tests are merely a way to write a spec. They are the same thing.

> The thing is it is very very difficult to know if your tests have complete coverage.

There is no way to avoid that, though. Like you point out, not even formal proofs, the closest speccing methodology we know of to try and avoid this, is immune.

> Tests are an approximation of your spec.

Specs are an approximation of what you actually want, sure, but that does not change that tests are the spec. There are other ways to write a spec, of course, but if you went down that road you wouldn't also have tests. That would be not only pointless, but a nightmare due to not having a single source of truth which causes all kinds of social (and sometimes technical) problems.

▲ godelski 5 days ago | parent [-]

  > that does not change that tests are the spec.

I disagree. It's, like you say, one description of your spec but that's not the spec.

  > not having a single source of truth

Well that's the thing, there is no single source of truth. A single source of truth is for religion, not code.

The point of saying this is to ensure you don't fall prey to fooling yourself. You're the easiest person for you to fool, after all. You should always carry some doubt. Not so much it is debilitating, but enough to keep you from being too arrogant. You need to constantly check that your documentation is aligned to your specs and that your specs are aligned to your goals. If you cannot see how these are different things then it's impossible to check your alignment and you've fooled yourself.

▲ 9rx 5 days ago | parent [-]

> You need to constantly check that your documentation is aligned to your specs

Documentation, tests, and specs are all ultimately different words for the same thing.

You do have to check that your implementation and documentation/spec/tests are aligned, which can be a lot of work if you do so by hand, but that's why we invented automatic methods. Formal verification is theoretically best (that we know of) at this, but a huge pain in the ass for humans to write, so that is why virtually everyone has adopted tests instead. It is a reasonable tradeoff between comfort in writing documentation while still providing sufficient automatic guarantees that the documentation is true.

> If you cannot see how these are different things

If you see them as different things, you are either pointlessly repeating yourself over and over or inventing information that is, at best, worthless (but often actively harmful).

▲ godelski 5 days ago | parent [-]

  > different words for the same thing

You're still misunderstanding and missing the layer of abstraction, which is what I'm (and others are) talking about

We have 3 objects: doc, test, spec. How do you prove they are the same thing?

You are arguing that they all point to the same address.

I'm arguing they all have the same parent.

I think it's pretty trivial to show that they aren't identical, so I'll give two examples (I'm sure you can figure out a few more trivial ones):

  1) the documentation is old and/or incorrect, therefore isn't aligned with tests. Neither address nor value are equivalent here.
  2) docs are written in natural language, tests are written in programming languages. I wouldn't say that the string "two" (or even "2") is identical to the integer 2 (nor the float 2). Duck typing may make them *appear* the same and they may *reference* the same abstraction (or even object!), but that is a very different thing than *being* the same. We could even use the classic Python mistake of confusing "is" with "==" (though that's a subset of the issue here).

Yes, you should simplify things as much as possible but be careful to not simplify further

▲ 9rx 5 days ago | parent [-]

> We have 3 objects: doc, test, spec. How do you prove they are the same thing?

You... don't? There is nothing good that can come from trying to understand crazy. Best to run away as fast as possible if you ever encounter this.

> You are arguing that they all point to the same address.

Oh? I did say if you document something the same way three different times (even if you give each time a different name, as if that somehow makes a difference), you are going to pointlessly end up with the same thing. I am not sure that necessarily equates to "the same address". In fact,

> I'm arguing they all have the same parent.

I also said that if they don't end up being equivalent documentation then you will only find difference in information that isn't useful. And that often that information becomes detrimental (see some of the adjacent comments that go into that problem). This is "having the same parent".

In reality, I "argued" both. You'd have better luck if you read the comments before replying.

> you should simplify things as much as possible but be careful to not simplify further

Exactly. Writing tests, documentation, or specs (whatever you want to call it; it all caries the same intent) in natural language certainly feels simpler in the moment, but you'll pay the price later. In reality, you at very least need a tool that supports automatic verification. That could mean formal verification, but, as before it's a beast that is tough to wrangle. More realistically, tests are going to be the best choice amid all the tradeoffs. Industry (including the Haskell fanbois, even) have settled on it for good reason.

> docs are written in natural language, tests are written in programming languages.

Technically "docs" is a concept of less specificity. Documentation can be written in natural language, that is true, but it can also be written in code (like what we call tests), or even pictures or video. "Tests" carries more specificity, being a particular way to write documentation — but ultimately they are the same thing. Same goes for "spec". It describes a different level of specificity (less specific than "tests", but more specific than "docs"), but not something entirely different. It is all documentation.

▲ godelski 5 days ago | parent [-]

  > In reality, I "argued" both.

I mean it is hard to have this conversation because you will say that they are the same thing and then leverage the fact that they aren't while disagreeing with me but using nearly identical settings to my examples.

I mean if your argument is that a mallard (test) and a muscovy (docs) are both types of ducks but a mallard is not a muscovy and a muscovy is not a mallard, then I fail to see how we aren't on the same page. I can't put it any clearer than this: all mallards are ducks but not all ducks are mallards. In other words, a mallard is a duck, but it is not representative of all ducks. You can't look at a mallard and know everything there is to know about ducks. You'll be missing stuff. If you treat your mallard and duck as isomorphic you're going to land yourself into trouble, even if most (domesticated) ducks are mallards.

It isn't that complex and saying "don't be overly confident" isn't adding crazy amounts of complexity that is going to overwhelm yourself. It's simply a recognition that you can't write a perfect spec.

Look, munificent[0] is saying the same thing. So is Kinrany[1], and manmal[2]. Do you think we're all wrong? In exactly the same way?

Besides, this whole argument is literally a demonstration of our claim. If you could write a perfect spec you'd (and we'd) be communicating perfectly and there'd be no hangup. But if that were possible we wouldn't need to write code in programming languages in the first place![3]

[0] https://news.ycombinator.com/item?id=44713138

[1] https://news.ycombinator.com/item?id=44713314

[2] https://news.ycombinator.com/item?id=44712266

[3] https://www.cs.utexas.edu/~EWD/transcriptions/EWD06xx/EWD667...

	▲	9rx 5 days ago \| parent [-]
		> I mean if your argument is that a mallard (test) and a muscovy (docs) are both types of ducks To draw a more reasonable analogy with how the words are actually used on a normal basis, you'd have fowl (docs), ducks (specs), and mallards (tests). As before, the terms change in specificity, but do not refer to something else entirely. Pointing at a mallard and calling it a duck, or fowl, doesn't alter what it is. It is all the very same animal. Yes, fowl could also refer to chickens just as documentation could refer to tax returns. 'Tis the nature of using a word lacking specificity. But from context one should be able to understand that we're not talking about tax returns here. But I don't have an "argument". High school debate team is over there. > It's simply a recognition that you can't write a perfect spec. That was recognized from the onset. What is the purpose of adding this again? > Do you think we're all wrong? We're all bad at communicating, if that's what you are straining to ask. Which isn't exactly much of a revelation. We've both already indicated as such, as have many commenters that came before us.