| I think what you're missing is that "you write tests so that you know your code works" doesn't actually work for some important classes of "works," security and concurrency (the subject of this HN discussion) being two prominent ones. That's because testing only shows that your code works for the cases you test. And, when it comes to security and concurrency, identifying the cases that matter is a hard enough problem that if you can figure out how to do it more reliably, that's publishable research. Think about it: If you're writing code and don't realize that it can deadlock under certain conditions, how are you going to realize that you need to test for whether it deadlocks under those conditions? If you're writing code that interpolates one kind of string into another and don't realize that you may have created an XSS vulnerability, are you suddenly going to gain that insight when you're writing the tests? |
| |
| ▲ | latchkey 4 days ago | parent [-] | | You run your code in production, you see it is deadlocking and you fix it. How do you fix it? You write a test the reproduces the deadlock, you fix the code and your test passes. Just like any other testing. I'm not arguing that you magically predict everything that's going to happen. But, without those tests and the culture of writing tests and writing your code to be testable, you're screwed when you do have bugs in production. | | |
| ▲ | tmoertel 4 days ago | parent [-] | | What you wrote was "you write tests so that you know your code works," but what you seem to have meant is "you write tests so that you when you get burned in production by problems that your tests didn't anticipate, you can write more tests at that time to make sure those newly discovered problems don't burn you again." That's nice, but it's far from "knowing your code works". When code works, it doesn't burn you in production. | | |
| ▲ | latchkey 4 days ago | parent [-] | | This nit pick is peak HN obstinacy. How do you know your code is going to work in production before you got there? You wrote tests. | | |
| ▲ | tmoertel 4 days ago | parent [-] | | You're missing something important. You know your code is going to work in production before you got there not because you wrote tests but because you thought about what it means for your code to work in production and then came up with a plan to generate the required confidence that it actually will work in production. And, for any nontrivial system, tests can only satisfy part of that plan. The goal isn't to have well-tested code. The goal is to have code that you can easily be confident will work as intended. And testing is only good at establishing some kinds of that confidence. For the other kinds, such as confidence you're not going to launch a bunch of security disasters or concurrency landmines, you need to do something else: types, proofs, correctness by construction, model checking, and so on. I wrote about this idea almost twenty years ago: https://blog.moertel.com/posts/2006-10-10-unit-testing-is-a-... I elaborated in a later post:
https://blog.moertel.com/posts/2012-04-15-test-like-youre-be... That you seem to believe otherwise is probably why a lot of people are having trouble with your claim that that world doesn't need better concurrency abstractions, just more tests because "if you write comprehensive unit tests, it is not easy to have bugs in golang." | | |
| ▲ | latchkey 4 days ago | parent [-] | | Not a single person has spoken up to say that they write a lot of tests AND they have a lot of bugs. All the negativity (downvotes) has come from people who are trying to argue that writing tests doesn't solve the problem of bugs. The same people who don't write a lot of tests AND have a lot of bugs. ¯\_(ツ)_/¯ I write a lot of tests and I don't have bugs. I have decades of experience and millions of lines of code, with this simple fact. I know it is true, at least for me. I don't know what else to bike shed here other than the constant downvoting by people who somehow don't believe my claim. The loss in karma doesn't bother me, I know I'm right on this and it appears as though the only people who disagree with me are the same people who don't write tests (and have a lot of bugs). golang is a relatively simple language. It is why I like it so much. Occasionally there are somewhat difficult things to reason about, but if you write golang code that is easily testable (and this requires thought and planning), then my experience is that even the "harder" channel/goroutine code can always be tested in one way or another. | | |
| ▲ | tmoertel 4 days ago | parent [-] | | > I know I'm right on this and it appears as though the only people who disagree with me are the same people who don't write tests (and have a lot of bugs). For the record, I write lots of tests, and don't have bugs. I even wrote a testing framework. Nobody is arguing that writing tests is dumb. The pushback is on your insistence that writing tests is all you need: > All the negativity (downvotes) has come from people who are trying to argue that writing tests doesn't solve the problem of bugs. Writing tests doesn't solve the problem of security bugs. Writing tests doesn't solve the problem of concurrency bugs. Writing tests to prove your code is bug free in those cases is expensive and error prone. People who care about these things know to go beyond testing when testing isn't enough. That's why things like model checkers exist. Nobody is arguing that tests are dumb. The argument is that if writing tests is all you're doing to get the bugs out of your code, you probably aren't very effective at preventing certain classes of problems. For instance: Show me the tests you'd write to prove your software doesn't have XSS vulnerabilities. | | |
| ▲ | latchkey 4 days ago | parent [-] | | > Show me the tests you'd write to prove your software doesn't have XSS vulnerabilities. I'd have tests around the code that renders 3rd party user input and integration tests for the display of the data. I've built some of the most heavily trafficked websites on the planet (porn), with user input (comments) and have never had an XSS issue. | | |
| ▲ | tmoertel 3 days ago | parent [-] | | Show me the tests. If you can't show me the tests you'd use to prove you don't have XSS problems, it's hard to believe that your tests are effective at preventing XSS problems. > I've built some of the most heavily trafficked websites on the planet (porn), with user input (comments) and have never had an XSS issue. Right, because the gold standard for proof in the security field is "we never had [read: noticed] an issue." | | |
| ▲ | latchkey 3 days ago | parent [-] | | It was code written in 2009 and private, not open source and I of course didn't take it with me when I left the company. I ran it for 4 years and we never had a single security incident. We took it very seriously. Partly because our code (in Java) was a rewrite from some really buggy PHP, that did in fact have a bunch of holes in it (and no testing). You're also being absurd. We started this talking about golang testing and it has somehow gone off the rails to me having to prove things to you about XSS? Come on, what is with the hostility? Is this how you treated people while working at Google? | | |
| ▲ | tmoertel 3 days ago | parent [-] | | I'm only asking you to show me how you'd write tests to detect XSS (or concurrency) problems. In Go or the language of your choice. You've claimed that writing tests as all you need. I'm asking you to show how it's done. Just in general. No need to share actual code you've written in the past. |
|
|
|
|
|
|
|
|
|
|
| > It is not a chore, it is our job. This is what we do. I'm not sure how to meaningfully distinguish the two here. I'm saying it takes extra effort, not denying that it's my job. It's non-trivial, that's all I'm trying to say. > How do you fix a production bug without breaking something else? You write tests so that you know your code works. Of course, you're right. Sometimes writing the tests can be harder than writing the code, though. |
