| |
| ▲ | Xylakant 19 hours ago | parent | next [-] | | It does, if you read the report and run a little (implied) math. The report says that their historical data gives them an estimate of 1000 Memory Safety issues per Million Lines of Code for C/C++. The same team currently has 5 Million lines of Rust code, of which 4% are unsafe (200 000). Assuming that unsafe Rust is on par with C/C++, this gives us an expected value of about 200 memory safety issues in the unsafe code. They have one. Either they have 199 hidden and undetected memory safety issues, or the conclusion is that even unsafe Rust is orders of magnitude better than C/C++ when it comes to memory safety. I trust them to track these numbers diligently. This is a seasoned team building foundational low level software. We can safely assume that the Android team is better than the average C/C++ programmer (and likely also than the average Rust programmer), so the numbers should generalize fairly well. Part of the benefits of Rust is indeed that it allows local reasoning about crucial parts of the code. This does allow for higher scrutiny which will find more bugs, but that's a result of the language design. unsafe {} was designed with that im mind - this is not a random emergent property. | | |
| ▲ | menaerus 19 hours ago | parent [-] | | They say "With roughly 5 million lines of Rust in the Android platform and one potential memory safety vulnerability found (and fixed pre-release), our estimated vulnerability density for Rust is 0.2 vuln per 1 million lines (MLOC).". Do you honestly believe that there is 1 vulnerability per 5 MLoC? | | |
| ▲ | Xylakant 19 hours ago | parent [-] | | 1 memory safety vulnerability, that's a pretty important distinction. Yes, I believe that at least the order of magnitude is correct because 4 800 000 of those lines are guaranteed to not have any by virtue of the compiler enforcing memory safety. So it's 1 per 200 000, which is 1-2 orders of magnitude worse, but still pretty darn good. Given that not all unsafe code actually has potential for memory safety issues and that the compiler still will enforce a pretty wide set of rules, I consider this to be achievable. This is clearly a competent team that's writing important and challenging low-level software. They published the numbers voluntarily and are staking their reputation on these reports. From personal observation of the Rust projects we work on, the results track with the trend. There's no reason for me to disbelieve the numbers put forward in the report. | | |
| ▲ | menaerus 19 hours ago | parent [-] | | 1 per 5M or 1 per 200K is pretty much unbelievable, especially in such a complex codebase, so all I can say then is to each their own. | | |
| ▲ | littlestymaar 17 hours ago | parent [-] | | > especially in such a complex codebase You accidentally put the finger on the key point, emphasis mine. When you have a memory-unsafe language, the complexity of the whole codebase impact your ability to uphold memory-related invariants. But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they shouldn't interact with other unsafe blocks in a different module. So the complexity related to one unsafe block is in fact contained to his own module, and doesn't spread outside. And that makes everything much more tractable since you never have to reason about the whole codebase, but only about a limited scope everytime. | | |
| ▲ | menaerus 6 hours ago | parent | next [-] | | No, this is just an example of confirmation bias. You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so. Google, for anyone unbiased, lost the credibility when they put this figure into the report. I wonder what was their incentive for doing so. > But unsafe block are, by definition, limited in scope and assuming you design your codebase properly, they shouldn't interact with other unsafe blocks in a different module. So the complexity related to one unsafe block is in fact contained to his own module, and doesn't spread outside. And that makes everything much more tractable since you never have to reason about the whole codebase, but only about a limited scope everytime. For anyone who has written low-level code with substantial complexity knows that this is just a wishful thinking. In such code, abstractions fall-apart and "So the complexity related to one unsafe block is in fact contained to his own module, and doesn't spread outside" is just wrong as I explained in my other comment here - UB taking place in unsafe section will transcend into the rest of the "safe" code - UB is not "caught" or put into the quarantine with some imaginative safety net at the boundary between the safe and unsafe sections. | | |
| ▲ | Xylakant 4 hours ago | parent | next [-] | | Let's take a simple example to illustrate how unsafe {} cuts down the review effort for many operations. Take a static mutable global variable (a global counter for example). Reading a static is safe, mutating it (increasing the counter) is not - it requires an unsafe {} block. If you need to check which places mutate this global static you only need to check the unsafe parts of the code - you know that no other part of your code could mutate it, the compiler won't let you. If you have a bug that is related to mutating this static, then it might manifest anywhere in your code. But you know for certain that the root cause must be in one of your unsafe blocks - even if you don't know which one. Good programming practice will cut down that effort even more by dictating that unsafe access should be grouped in modules. For example when binding to a C module (unsafe) you'd usally generate an unsafe wrapper with bindgen and then write a safe wrapper on top of that. Any access that tries to go around the safe wrapper would be frowned upon and likely fail review. And again, the compiler will help you there: Any access that tries to bypass the safe api would need to be unsafe {} again and automatically receive extra scrutiny in a review, making it less likely to slip through. Compare that to a C codebase where anything goes. A static might be mutated anywhere in your codebase, even through a pointer to it - meaning you can't even reliably grep for it. It may slip through review unnoticed because no attention is drawn to it and cause bugs that are hard to trace and reason about. If you're writing embedded code, similar considerations apply - access to registers etc. require unsafe {}. But because access is unsafe {}, it's usually gated behind a safe api that is the boundary of the low-level code and the higher buisness logic. Unsurprisingly, these are critical parts of the code - hence they receive extra scrutiny and in our project, we allocate substantial review capacity on those. And the compiler will enforce that no safe code can circumvent the access layer. The number you're tagging as unrealistic figure is the result of dedicated and careful design of language and compiler features to achieve exactly this outcome. It's not a random fluke, very clever people did sit down and thought about how to achieve this. | |
| ▲ | sunshowers 4 hours ago | parent | prev | next [-] | | You sound like my teacher in school, after my friend submitted an assignment so good my teacher thought he was cheating. | |
| ▲ | littlestymaar 6 hours ago | parent | prev [-] | | > You're given a totally unrealistic figure of 1 vuln per 200K/5M LoC and now you're hypothesizing why that could be so. You are the one claiming it's unrealistic. And you gave zero argument why besides “the codebase is complex”, which I refuted. See the definition of complexity: > The term is generally used to characterize something with many parts where those parts interact with each other in multiple ways, culminating in a higher order of emergence greater than the sum of its parts Each unsafe block may be “difficult” in itself, but the resulting system isn't “complex” because you don't have this compounding effect. > I wonder what was their incentive for doing so. And obviously it must be malice… > For anyone who has written low-level code with substantial complexity knows that this is just a wishful thinking. In such code, abstractions fall-apart and "So the complexity related to one unsafe block is in fact contained to his own module, and doesn't spread outside" is just wrong as I explained in my other comment here - UB taking place in unsafe section will transcend into the rest of the "safe" code - UB is not "caught" or put into the quarantine with some imaginative safety net at the boundary between the safe and unsafe sections. I think you don't understand the problem as well as you think you do. Of course if the UB happens then all bets are off! Its consequences won't be limited to a part of the code, by definition. And nobody said otherwise. But for the UB to happen, there must be some violation of an memory invariant (the most common would be using a value after free, freeing twice, accessible the same memory from multiple threads without synchronization or, and this is specific to Rust, violating reference aliasing rules). To avoid violating these invariants, the programmer must have a mental model of the ownership over all the system on which these invariants apply. For C or C++, it means having a mental model of all the code base, because the invariants related to one piece of code can be violated from everywhere. In Rust this is different, you're not going to have raw pointers to one piece of data being used in multiple parts of the code (well, if you really want, nobody stops you, but I'm confident the Android team didn't). And as such, you'll have to think about the invariants only at the scale of one module. Building an accurate mental model of a 350-line module is much more tractable for a human than doing the same for an entire codebase, and it's not even close. |
| |
| ▲ | Xylakant 17 hours ago | parent | prev [-] | | That's the other interesting observation you can draw from that report. The numbers contained in the first parts about review times, rollback rates, etc. are broken down by change size. And the gap widens for larger changes. This indicates that Rusts language features support reasoning about complex changesets. It's not obviously clear to me which features are the relevant ones, but my general observation is that lifetimes, unsafe blocks, the borrow checker allow people to reason about code in smaller chunks. For example knowing that there's only one place where a variable may be mutated supports understanding that at the same time, no other code location may change it. |
|
|
|
|
| |
| ▲ | baq 20 hours ago | parent | prev | next [-] | | literally from the quote: unsafe{} doesn't actually disable all or even most of Rust’s safety checks (a common misconception).
| | |
| ▲ | menaerus 19 hours ago | parent [-] | | They also say The practice of encapsulation enables local reasoning about safety invariants.
which is not fully correct. Undefined behavior in unsafe blocks can and will leak into the safe Rust code so there is nothing there about the "local reasoning" or "encapsulation" or "safety invariants".This whole blog always read to me as too much like a marketing material disguised with some data so that it is not so obvious. IMHO | | |
| ▲ | aw1621107 18 hours ago | parent | next [-] | | > which is not fully correct. Undefined behavior in unsafe blocks can and will leak into the safe Rust code so there is nothing there about the "local reasoning" or "encapsulation" or "safety invariants". Strictly speaking, that encapsulation enables local reasoning about safety invariants does not necessarily imply that encapsulation guarantees local reasoning about safety invariants. It's always possible to write something unadvisable, and no language is capable of preventing that. That being said, I think you might be missing the point to some extent. The idea behind the sentence is not to say that the consequences of a mistake will not be felt elsewhere. The idea is that when reasoning about whether you're upholding invariants and/or investivating something that went wrong, the amount of code you need to look at is bounded such that you can ignore everything outside those bounds; i.e., you can look at some set of code in complete isolation. In the most conservative/general case that boundary would be the module boundary, but it's not uncommon to be able to shrink those boundaries to the function body, or potentially even further. This general concept here isn't really new. Rust just applied it in a relatively new context. | | |
| ▲ | menaerus 17 hours ago | parent [-] | | Yes, but my point is when things blow up how exactly do you know which unsafe block you should look into? From their statement it appears as if there's such a simple correlation between "here's your segfault" and "here's your unsafe block that caused it", and which I believe there isn't, and which is why I said there's no encapsulation, local reasoning etc. | | |
| ▲ | Xylakant 17 hours ago | parent | next [-] | | But you know that it's one of them. That cuts 96% of their codebase out, even assuming you have no idea which one it could be. | |
| ▲ | aw1621107 16 hours ago | parent | prev [-] | | > Yes, but my point is when things blow up how exactly do you know which unsafe block you should look into? In the most general case, you don't. But again, I think that that rather misses the point the statement was trying to get at. Perhaps a more useful framing for you would be that in the most general case the encapsulation and local reasoning here is between modules that use unsafe and everything else. In some (many? most?) cases you can further bound how much code you need to look at if/when something goes wrong since not all code in unsafe modules/functions/blocks depend on each other, but in any case the point is that you only need to inspect a subset of code when reasoning about safety invariants and/or debugging a crash. > From their statement it appears as if there's such a simple correlation between "here's your segfault" and "here's your unsafe block that caused it", I don't get that sense from the statement at all. | | |
| ▲ | menaerus 6 hours ago | parent [-] | | > in the most general case the encapsulation and local reasoning here is between modules that use unsafe and everything else This would be the same narrative as in, let's say, C++. Wrap the difficult and low-level memory juggling stuff into "modules", harden the API, return the references and/or smart-pointers, and then just deal with the rest of the code with ease, right? Theoretically possible but practically impossible. First reason is that abstractions get really leaky, and they especially get really leaky in the code that demands the upmost performance. Anyone who implemented their own domain/workload specific hash-map or mutex or anything similarly foundational will understand this sentiment. Anyway, if we just have a look into the NVMe driver above, there're no "unsafe modules". Second, and as I already argued, UB in the module library transcends into the rest of your code so I fail to understand how is it so that the dozens of unsafe sections make the reasoning or debugging any more simpler when reasoning is actually not a function of number of unsafe sections but it is the function of interactions between different parts of the code that end up touching the memory in the unsafe block in a way that it was not anticipated. This is almost always the case when dealing with undefined behavior. > I don't get that sense from the statement at all. It is a bit exaggerated example of mine but I do - their framing suggests ~exactly that and which is simply not true. |
|
|
| |
| ▲ | 18 hours ago | parent | prev [-] | | [deleted] |
|
| |
| ▲ | sophacles 20 hours ago | parent | prev [-] | | It actually does support it. Human attention is a finite resource. You can spend a little bit if attention in every line to scrutinize safety or you can spend a lot of time scrutinizing the places where you can't mechanically guarantee safety. It's safer because it spends the human attention resource more wisely. |
|
