Remix.run Logo
chadaustin 5 days ago

Every time this conversation comes up, I'm reminded of my team at Dropbox, where it was a rite of passage for new engineers to introduce a segfault in our Go server by not synchronizing writes to a data structure.

Swift has (had?) the same issue and I had to write a program to illustrate that Swift is (was?) perfectly happy to segfault under shared access to data structures.

Go has never been memory-safe (in the Rust and Java sense) and it's wild to me that it got branded as such.

tptacek 5 days ago | parent | next [-]

Right, the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term. People talk as if "memory safety" was a PLT axiom. It's not; it's a software security term of art.

This is just two groups of people talking past each other.

It's not as if Go programmers are unaware of the distinction you're talking about. It's literally the premise of the language; it's the basis for "share by communicating, don't communicate by sharing". Obviously, that didn't work out, and modern Go does a lot of sharing and needs a lot of synchronization. But: everybody understands that.

moefh 5 days ago | parent | next [-]

I agree that there are two groups here talking past each other. I think it would help a lot to clarify this:

> the issue here is that the "Rust and Java sense" of memory safety is not the actual meaning of the term

So what is the actual meaning? Is it simply "there are no cases of actual exploited bugs in the wild"?

Because in another comment you wrote:

> a term of art was created to describe something complicated; in this case, "memory safety", to describe the property of programming languages that don't admit to memory corruption vulnerabilities, such as stack and heap overflows, use-after-frees, and type confusions. Later, people uninvolved with the popularization of the term took the term and tried to define it from first principles, arriving at a place different than the term of art.

But type confusion is exactly what has been demonstrated in the post's example. So what kind of memory safety does Go actually provide, in the term of art sense?

tptacek 5 days ago | parent [-]

It's a contrived type confusion bug. It reads 42h because that address is hardcoded, and it does something that ordinary code doesn't do.

If you were engaged to do a software security assessment for an established firm that used Go (or Python, or any of the other mainstream languages that do shared-memory concurrency and don't have Rust's type system), and you said "this code is memory-unsafe", showing them this example, you would not be taken seriously.

If people want to make PLT arguments about Rust's correctness advantages, I will step out of the way and let them do that. But this article makes a security claim, and that claim is in the practical sense false.

ralfj 4 days ago | parent | next [-]

> It reads 42h because that address is hardcoded,

It is trivial to change this example into an arbitrary int2ptr cast.

> Go (or Python, or any of the other mainstream languages that do shared-memory concurrency and don't have Rust's type system),

As the article discusses, only Go has this issue. Python and Java and JavaScript and so on are all memory-safe. Maybe you are mixing up "language has data races" and "data races can cause the language itself to be broken"?

> If people want to make PLT arguments about Rust's correctness advantages, I will step out of the way and let them do that. But this article makes a security claim, and that claim is in the practical sense false.

This article makes a claim about the term "memory safety". You are making the claim that that's a security term. I admit I am not familiar with the full history of the term "memory safety", but I do know that "type safety" has been used in PLT for many decades, so it's not like all "safety" terms are somehow in the security domain.

I am curious what your definition of "memory safety" is such that Go satisfies the definition. Wikipedia defines it as

> Memory safety is the state of being protected from various software bugs and security vulnerabilities when dealing with memory access, such as buffer overflows and dangling pointers.

My example shows that Go does not enforce memory safety according to that definition -- and not through some sort of oversight or accident, but by design. Out-of-bounds reads and writes are possible in Go. The example might be contrived, but the entire point of memory safety guarantees is that it doesn't matter how contrived the code is.

I'm completely fine with Go making that choice, but I am not fine with Go then claiming to be memory safe in the same sense that Java or Rust are, when it is demonstrably not the case.

tptacek 4 days ago | parent [-]

The problem isn't that you couldn't hardcode a scarier value; it's that you have to demonstrate a plausible scenario in realistic code where an attacker controls both the value and the address it's written to.

While you're wondering why I keep claiming Go is a memory-safe language, you can also go ask the ISRG, which says the same thing I am at (checks notes) https://www.memorysafety.org/.

ralfj 4 days ago | parent [-]

> While you're wondering why I keep claiming Go is a memory-safe language, you can also go ask the ISRG, which says the same thing I am at

And yet Go violates the definition they give -- it doesn't prevent out-of-bounds accesses. (And just to be sure we're talking about the same thing, I'm specifically talking about Go here. All the other languages on their list are actually memory safe, as far as I know.)

> you have to demonstrate a plausible scenario in realistic code where an attacker controls both the value and the address it's written to.

So your definition of memory safety includes some notion of "plausible" and "realistic"? Neither https://www.memorysafety.org/docs/memory-safety/ nor Wikipedia have such a qualification in their definition. It would help if you could just spell out your definition in full, rather than having us guess.

4 days ago | parent | next [-]
[deleted]
HAL3000 4 days ago | parent | prev | next [-]

> So your definition of memory safety includes some notion of "plausible" and "realistic"? Neither https://www.memorysafety.org/docs/memory-safety/ nor Wikipedia have such a qualification in their definition. It would help if you could just spell out your definition in full, rather than having us guess.

This is a strawman argument, you're arguing semantics here. You're a smart person, so you know exactly what he means. The perception created by your article is that people shouldn't use Go because it's not memory-safe. But the average developer hearing "not memory-safe" thinks of C/C++ level issues, with RCEs everywhere.

Unless you can show a realistic way this could be exploited for RCE in actual programs, you're just making noise. Further down the thread, you admit yourself that you're in a PLT research bubble and it shows.

dcow 4 days ago | parent | next [-]

You definitely shouldn’t use Go, but it’s not because of the discussion in TFA. I jest, rhetorically.

Seriously, why are we bashing a researcher for being academic? This makes no fucking sense. Nobody claimed anywhere that people should stop using Go.

esoterae 4 days ago | parent | prev | next [-]

Did you not notice how this started over someone saying "That's not the definition of memory safety" and then prevaricating about the bush when asked to provide their definition? Your theory that this is an argument over semantics is correct, but not fully understood.

ralfj 4 days ago | parent | prev | next [-]

> The perception created by your article is that people shouldn't use Go because it's not memory-safe.

Uh, where exactly am I saying or implying that? I am, in fact, saying that Go is much closer to memory-safe languages than to C, safety-wise.

But I am arguing that the term "memory safe" should only be used for languages that actually went through the effort of thinking this problem through to the end and plugging all the holes through which memory safety violates can sneak in. Go is 99% there, but it's falling slightly short of the goal. I think that's a useful distinction, and I am disappointed that it is regularly swept under the rug, which is why I wrote this blog post. You are free to disagree, I never expected to convince everyone. But I think I gave some people some new food for thought, and that's all I can hope for.

hdevalence 4 days ago | parent | prev [-]

> you're arguing semantics here

Yes, semantics — what do things mean, exactly? — is the subject of the discussion here and is actually quite important in general.

tptacek 4 days ago | parent | prev [-]

You're just wrong about this. The ability to write contrived code that does an out-of-bounds write, or to induce crashes, doesn't violate the notion of "memory safety" as an ordinary term of art.

ralfj 4 days ago | parent | next [-]

Yeah I understand that that's how you like to use the term, you've been very clear about that. What I am curious about is whether that's just you. Because the source you gave last time, https://www.memorysafety.org/docs/memory-safety/, doesn't agree with what you are saying, and neither does Wikipedia.

I am honestly curious here. I am a PLT researcher so I am in a bubble where people use the term consistently with how I use it. You are the first person I meet (for some notion of "meet" ;) that uses the term differently. But without external sources it's hard to judge how wide-spread your definition (that you still haven't spelled out...) is.

tptacek 4 days ago | parent | next [-]

Again: my usage of the term is widespread enough that the ISRG uses it to refer to Go as well, as does, well, basically everybody else in the industry. I think you've just message-boarded yourself into believing this is a live debate. There is no sequence of words you're going to come up with to convince me that everybody is wrong when they say "Go is a memory safe language".

ralfj 4 days ago | parent [-]

You keep making arguments by assertion without giving sources, so :shrug: yeah this isn't going to go anywhere.

I think we actually agree on all of the factual points here, we just don't agree on how languages should be categorized/labeled according to their guarantees in both a theoretical and a practical sense, and that's largely a subjective matter anyway. So, happy to agree to disagree here.

Thaxll 4 days ago | parent | prev [-]

Go is memory safe, what do you think of: https://www.nsa.gov/Press-Room/Press-Releases-Statements/Pre...

U.S. and International Partners Issue Recommendations to Secure Software Products Through Memory Safety

They recommand Go among other language in their paper.

https://media.defense.gov/2023/Dec/06/2003352724/-1/-1/0/THE...

ralfj 4 days ago | parent [-]

Yeah, Go is often listed with memory-safe languages, I know that. And yet when people define memory safety, Go usually fails to satisfy that definition. That's why I was asking for a definition of memory safety that would include Go.

kiitos 2 days ago | parent [-]

I suppose Go's notion of memory safety is satisfied by forbidding pointer arithmetic, and, maybe somewhat transitively, preventing arbitrary out-of-bounds access to memory. It definitely satisfies this notion of memory safety. Maybe this notion of memory safety is not considered to be correct, or relevant, or whatever, by whomever. That's fine.

rowanG077 4 days ago | parent | prev | next [-]

Did you consider that the organization can be wrong?

> Memory safety is a property of some programming languages that prevents programmers from introducing certain types of bugs related to how memory is used. Since memory safety bugs are often security issues, memory safe languages are more secure than languages that are not memory safe.

That is the definition they give. Since Go does not "prevent programmers from introducing certain types of bugs related to how memory is used." it does not fall under this definition. They can list go as memory safe, but then either they disagree with their own definition or made the mistake of adding Go to that list. Memory safety is not a spectrum. You are either memory safe or unsafe. The spectrum is in the unsafety. Go is obviously less unsafe than C for example.

kiitos 2 days ago | parent [-]

> Memory safety is not a spectrum. You are either memory safe or unsafe.

if there is one takeaway from this discussion, i think it must be that memory safety does not have any single, commonly-accepted, or objective definition -- and that it is pretty obviously a spectrum, not a boolean

rowanG077 19 hours ago | parent [-]

I could agree to that if the people who claim it would at least put it in their definition. But as it stands it indeed is a boolean. You have to give a definition of something like:

"Memory safety denotes the degree to which a programming language guides and protects developers from memory‑related errors—ranging from minimal, manual checks to comprehensive static and runtime enforcement—through mechanisms like strong typing, ownership or borrow checking, and garbage collection."

And then also include modern C++ in their lists. because by all accounts it is memory safe by that definition.

zozbot234 4 days ago | parent | prev [-]

Then the "security" definition is totally useless, because even C can be memory safe. What about pointers, malloc(), free(), unchecked enums etc. etc.? Oh, those are just "contrived" language features you're not really supposed to use. You can write FORTRAN in any language!

tptacek 4 days ago | parent [-]

C is the archetypical memory-unsafe language. When you've reached the point where you're simultaneously arguing that C is memory-safe and Go isn't, you should recognize you've made a wrong turn somewhere.

zozbot234 4 days ago | parent [-]

My point with those comparisons is that you have not bothered to define a reasonable and verifiable standard for what counts as "contrived" code - which is what ultimately seems to determine whether a language is memory safe, according to your definition.

nicoburns 4 days ago | parent | prev | next [-]

You are however replying to thread where a Dropbox engineer calls it "a right of passage" to introduce such bugs to their codebase. Which suggests that it is by no means unheard of for these problems to crop up in real-world code.

tptacek 4 days ago | parent | next [-]

Again: introducing surprising correctness bugs? Crashing programs? Absolutely. I don't know how many different ways I can say that my concern here is the misuse of a security term of art. Dropbox engineers do not have as a rite of passage introducing or finding RCE vulnerabilities in Go code. Would that it were so! My job would be much more interesting.

zozbot234 4 days ago | parent [-]

> correctness bugs? Crashing programs? Absolutely.

Denial of service can absolutely be a security issue, as can any correctness bug if it leads to unintended behavior or corrupted data.

tptacek 4 days ago | parent [-]

If that's where we're at, where unhandled exceptions are the security issues we're hanging on, I'll consider my argument won.

zozbot234 4 days ago | parent [-]

That might be a reasonable argument if you were guaranteed an unhandled exception in this instance. Unfortunately that's not the case.

tptacek 4 days ago | parent [-]

If you could demonstrate something better than that, we wouldn't be arguing about the severity of DOS attacks.

samus 4 days ago | parent | prev | next [-]

Doesn't Dropbox write a lot of Python extensions in C for speedup?

pclmulqdq 4 days ago | parent [-]

Excuse me, but in this thread we are bashing go, not making logical arguments.

ioasuncvinvaer 4 days ago | parent [-]

What is the argument?

pclmulqdq 4 days ago | parent [-]

That Dropbox is very happy with unsafe languages despite the top-level comment.

blub 4 days ago | parent | prev [-]

Many FAANG & co engineers are overrated. If every new hire is introducing concurrency bugs in a Golang codebase, refactor, do better review and maybe use concurrency questions instead of leetcode.

I’ll take tptacek’s word over most FAANG type on such topics if we’re doing appeals to authority. The guy is very practical, unlike the Rust community which is incredibly focused on theoretical correctness instead of real-world experiences.

stickfigure 5 days ago | parent | prev | next [-]

I have recently come to the conclusion that everything I ever thought was "contrived" is currently standard practice in some large presently existing organization.

tptacek 5 days ago | parent [-]

Take that to the Apple bounty program with your crasher bug and tell them they should pay out as if you'd confirmed RCE, see how it goes. This is an engineering question; it's not about vibes.

It's not even always the case that corrupted data structures (or even pointers) in C code are exploitable. You need attacker control of data and where it goes in memory. It's far less often the case in Python or Go --- in fact, it's basically never the case. As evidence for that claim: the zero memory corruption RCEs in all of shipping Go code, of which there is a lot.

NitpickLawyer 5 days ago | parent [-]

Dunno about Apple, but goog sometimes pays out bugs that are "theoretical" in the way you describe. That is, you show that there's a bug somewhere, but you can't "reach" it from user data. They'll pay less than a PoC, obviously, but will pay. YMMV, etc.

jacquesm 4 days ago | parent | prev [-]

My own takeaway after looking at corporate codebases for four decades is that the state of the art in software development at banks, governments, insurance companies, airlines, health care and so on is such that I long for the time before the internet.

Sure, those mainframes from the 80's weren't bullet proof either. But you first had to get to them. And even if the data traveled in plain text on leased lines (point-to-point but not actually point-to-point (that would require a lot of digging), no multiplexing) you had to physically move to the country where they were located to eavesdrop on them, and injecting data into the stream was a much harder problem.

Ygg2 5 days ago | parent | prev | next [-]

> People talk as if "memory safety" was a PLT axiom. It's not; it's a software security term of art.

It's been in usage for PLT for at least twenty years[1]. You are at least two decades late to the party.

    Software is memory-safe if (a) it never references a memory location outside the address space allocated by or that entity, and (b) it never executes intstruction outside code area created by the compiler and linker within that address space.
[1]https://llvm.org/pubs/2003-05-05-LCTES03-CodeSafety.pdf
zbentley 5 days ago | parent | next [-]

Not GP, but that definition seems not to be the one in use when describing languages like Rust--or even tools like valgrind. Those tools value a definition of "memory safety" that is a superset (a big one) of the definition referenced in that paper: safety as preventing incorrect memory accesses within a program, regardless of whether those accesses are out of bounds/segmentation violations.

adgjlsfhk1 4 days ago | parent [-]

it's not, but for a very subtle reason. To prove memory safety, you need to know that the program never encounters UB (since at that point you have nothing known about the program)

Wowfunhappy 5 days ago | parent | prev [-]

...by that definition, can a C program be memory safe as long as it doesn't have any relevant bugs, despite the choice of language? (I realize that in practice, most people are not aware of every bug that exists in their program.)

cibyr 5 days ago | parent | next [-]

Can a C program be memory safe as long as it doesn't have any relevant bugs? Yes, and you can even prove this about some C programs using tools like CBMC.

Waterluvian 5 days ago | parent | prev | next [-]

This is way outside my domain but isn’t the answer: yes, if the code is formally proven safe?

Doesn’t NASA have an incredibly strict, specific set of standards for writing safety critical C that helps with writing programs that can be formalized?

okanat 5 days ago | parent | next [-]

There are safety recommendations / best practice standards like CERT. None of them will prevent you from making intentional looking but logically unsound memory unsafe operations with C and C++. The code can be very indistinguishable from safe code. The things that C and C++ allow you to do basically makes code written in those languages impossible to fully formally prove. Although there are subsets, the basic integer operations and primitive types are messed up with C. So without uprooting how basic integer and pointer types work, it is impossible to make C and C++ safer. Such change will make all C and C++ programs invalid.

C and C++ always defaults to minimum amount of safety for maximum allowance of the compiler interpretation. The priority of the language designers of them is keeping existing terrible code running as long as possible first, letting compilers interpret the source code as freely as possible second.

That's why many military and aerospace code actually uses much safer and significantly more formally verifiable Ada.

uecker 5 days ago | parent | next [-]

There is also a lot of formally verified in C.

spookie 4 days ago | parent | prev | next [-]

> impossible to formally prove

If you assume the entire lang, yes. If you use a large subset, no. Furthermore, compiler interpretation might actually be sane! There are more compilers out there than GCC, Clang or MSVC. I suspect many assumptions are being made on this claim.

Waterluvian 5 days ago | parent | prev [-]

Thanks for explaining that. It really fills in a lot for me.

arto 4 days ago | parent | prev [-]

https://david-haber.github.io/posts/the-right-stuff/

ratmice 5 days ago | parent | prev | next [-]

It just seems like a bad definition (or at least ambiguous), it should say "cannot", or some such excluding term. By the definition as given if a program flips a coin and performs an illegal memory access, are runs where the access does not occur memory safe?

Ygg2 4 days ago | parent | prev | next [-]

Sure. It can be. In the same way, a C program can be provably correct. I.e., for all inputs it doesn't exhibit unexpected behavior. Memory safety and correctness are properties of the program being executed.

But a memory-safe program != memory safe language. Memory safe language helps you maintain memory-safety by reducing the chances to cause memory unsafety.

tremon 3 days ago | parent | prev [-]

There is a huge difference between "a C program can be memory safe if it is proven to be so by an external static analysis tool" and "the Java/Rust language is memory safe except for JNI resp unsafe sections of the code".

socalgal2 5 days ago | parent | prev | next [-]

> But: everybody understands that.

Everybody does not understand that otherwise there would be zero of these issues in shipping code.

This is the problem with the C++ crowd hoping to save their language. Maybe they'll finally figure out some --disallow-all-ub-and-be-memory-safe-and-thread-safe flag but at the moment it's still insanely trivial to make a mistake and return a reference to some value on the stack or any number of other issues.

The answer can not be "just write flawless code and you'll never have these issues" but at the moment that's all C++, and Go, from this article has.

blub 4 days ago | parent | next [-]

This comment highlights a very important philosophical difference between the Rust community and the communities of other languages:

- in other languages, it’s understood that perhaps the language is vulnerable to certain errors and one should attempt to mitigate them. But more importantly, those errors are one class of bug and bugs can happen. Set up infra to detect and recover.

- in Rust the code must be safe, must be written in a certain way, must be proven correct to the largest extent possible at compile time.

This leads to the very serious, solemn attitude typical of Rust developers. But the reality is that most people just don’t care that much about a particular type of error as opposed to other errors.

zozbot234 4 days ago | parent | next [-]

> ... it's understood that perhaps the language is vulnerable to certain errors and one should attempt to mitigate them. But more importantly, those errors are one class of bug and bugs can happen. Set up infra to detect and recover.

> in Rust the code must be safe, must be written in a certain way, must be proven correct to the largest extent possible at compile time.

Only for the Safe Rust subset. Rust has the 'unsafe' keyword that shows exactly where the former case does apply. (And even then, only for possible memory unsoundness. Rust does not attempt to fix all possible errors.)

blub 3 days ago | parent | next [-]

Well yes, only in the safe subset, but the safe subset is the alpha and omega of Rust.

Woe to those that use unsafe, for the scorn of the community shall be upon them. :)

pclmulqdq 4 days ago | parent | prev [-]

A not-so-secret secret of Rust is that liberal use of 'unsafe' is pretty much required for certain classes of high-performance code.

aystatic 4 days ago | parent [-]

imo if you're sprinkling around `unsafe` in your codebase "liberally", you're holding it wrong. In general it's really not that hard to encapsulate most unsafety into a wide-contract abstraction; I’d argue where Rust really shines is when you take advantage of the type system and static analyzer to automatically uphold invariants for you

J_Shelby_J 4 days ago | parent | prev [-]

> This leads to the very serious, solemn attitude typical of Rust developers.

Opposite really. I like rust because I can be care free and have fun.

tptacek 5 days ago | parent | prev [-]

Again: if you want to make that claim about correctness bugs, that's fine, I get it. But if you're trying to claim that naive Go code has memory safety security bugs: no, that is simply not true.

saurik 5 days ago | parent | next [-]

I cannot find anyone in this thread (nor in the article) making the claim you are arguing against, though... the reason for the example isn't "this demonstrates all Go code is wrong", but merely that "you can't assume that all Go code is correct merely because it is written in Go"; now, most code written in Go might, in fact, be safe, and it might even be difficult to write broken Go code; but, I certainly have come across a LOT of people who are claiming that we don't even have to analyze their code for mistakes because it is written in Go, which is not the case, because people do, in fact, share stuff all over the place and Go, in fact, doesn't prevent you from all possible ways of writing broken code. To convince these people that they have to stop making that assumption requires merely any example of code which fails, and to those people these examples are, in fact, elucidating. Of course, clearly, this isn't showing a security issue, but it isn't claiming to be; and like, obviously, this isn't something that would be sent to a bug bounty program, as who would it even be sent to? I dunno... you seem to have decided you want to win a really minute pedantic point against someone who doesn't exist, and it makes this whole thing very confusing.

ngrilly 4 days ago | parent [-]

The terms correctness (from a PLT perspective) and safety (from a security perspective) are not equivalent and interchangeable. I see them mixed up too much in this discussion.

ralfj 4 days ago | parent [-]

PLT has used the term "type safety" for a very long time -- so "safety" does not imply a security perspective. And yes it is indeed very different from correctness. But the article doesn't claim that memory safety should imply correctness -- that would be ridiculous, obviously you can write buggy programs in memory-safe languages. The article claims that Go is not memory-safe.

ngrilly 4 days ago | parent [-]

I was referring to comments mentioning correctness and safety as interchangeable terms. The article doesn’t mix them up.

ralfj 4 days ago | parent [-]

Fair, I misunderstood then. :)

const_cast 3 days ago | parent | prev [-]

I'm not understanding - if you're able to produce segfaults by, presumably, writing to memory out of bounds, what's stopping a vulnerability? Surely, if I can write past an array's bounds, then I can do a buffer overflow of some sort in some situations?

dev_l1x_be 4 days ago | parent | prev | next [-]

> But: everybody understands that.

I had to convince Go people that you can segfault with Go. Or you mean the language designers with using everybody?

pclmulqdq 4 days ago | parent [-]

You can segfault in Rust, too - there's a whole subset of the language marked "unsafe" that people ignore when making "safe language" arguments. The question is how difficult is it to have a segfault, and in Go it's honestly pretty hard. It's arguably harder in Rust but it's not impossible.

ralfj 3 days ago | parent [-]

It's impossible in safe Rust (modulo compiler bugs and things like using a debugger to poke in the program's memory from the outside). That's the key difference.

Of course unsafe Rust is not memory safe. That's why it is called like that. :) Go has unsafe operations too (https://go.dev/ref/spec#Package_unsafe), and of course if you use those all bets are off. But as you will notice, my example doesn't use those operations.

blub 4 days ago | parent | prev [-]

The problem Rust has is that it’s not enough to be memory safe, because lots of languages are memory safe and have been for decades.

Hence the focus on fearless concurrency or other small-scale idioms like match in an attempt to present Rust as an overall better language compared to other safe languages like Go, which is proving to be a solid competitor and is much easier to learn and understand.

zozbot234 4 days ago | parent [-]

Except that Swift also has safe concurrency now. It's not just Rust. Golang is actually a very nice language for problems where you're inherently dependent on high-performance GC and concurrency, so there's no need to present it as "better" for everything. Nevertheless its concurrency model is far from foolproof when compared to e.g. Swift.

blub 3 days ago | parent [-]

But that’s bad news for Rust adoption… Worst case for Rust is it takes just some (not all) marketshare from C and C++, because Swift, Golang, Java, Python, TypeScript, etc have cornered the rest.

junebash 5 days ago | parent | prev | next [-]

Swift is in the process of fixing this, but it’s a slow and painful transition; there’s an awful lot of unsafe code in the wild that wasn’t unsafe until recently.

RetpolineDrama 5 days ago | parent | next [-]

Swift 6 is only painful if you wrote a ton of terrible Swift 5, and even then Swift 5 has had modes where you could gracefully adopt the Swift 6 safety mechanisms for a long time (years?)

~130k LoC Swift app was converted from 5 -> 6 for us in about 3 days.

jamil7 5 days ago | parent | next [-]

Yes and no, our app is considerably larger than 130k LoC. While we’ve migrated some modules there are some parts that do a lot of multithreaded work that we probably will never migrate because they’d need to essentially be rewritten and the tradeoff isn’t really worth it for us.

isodev 4 days ago | parent | prev [-]

It's also painful if you wrote good Swift 5 code but now suddenly you need to closely follow Apple's progress on porting their own frameworks, filling your code base with #if and control flow just to make the compiler happy.

ardit33 5 days ago | parent | prev | next [-]

It is still incomplete and a mess. I don't think they thought through the actual main cases Swift is used for (ios apps), and built a hypothetical generic way which is failing on most clients. Hence lots of workarounds, and ways to get around it (The actor system). The isolated/nonisolated types are a bit contrived and causing real productivity loss, when the old way was really just 'everything ui in main thread, everything that takes time, use a dispatch queue, and call main when done'.

Swift is strating to look more like old java beans. (if you are old enough to remember this, most swift developers are too young). Doing some of the same mistakes.

Anways https://forums.swift.org/t/has-swifts-concurrency-model-gone... Common problems all devs face: https://www.massicotte.org/problematic-patterns

Anyways, they are trying to reinvent 'safe concurrency' while almost throwing the baby with the bathwater, and making swift even more complex and harder to get into.

There is ways to go. For simple apps, the new concurrency is easy to adopt. But for anything that is less than trivial, it becomes a lot of work, to the point that it might not make it worth it.

pjmlp 4 days ago | parent | next [-]

Their goal was always to be able to evolve to the point of being able fully replace C, Objective-C and C++ with Swift, it has been on their documentation and plenty of WWDC sessions since the early days.

isodev 4 days ago | parent | prev [-]

You're getting downvoted but I fully agree. The problem with Swift's safety has now moved to the tooling. While your code doesn't fail so often at runtime (still does, because the underlying system SDKs are not all migrated), the compiler itself often fails. Even the latest developer snapshot with Swift 6.2 it's quite easy to make it panic with just... "weird syntax".

A much bigger problem I think are the way concurrency settings are provided via flags. It's no longer possible to know what a piece of code does without knowing the exact build settings. For example, depending on Xcode project flags, a snippet may always run on the main loop, or not at all or on a dedicated actor all together.

A piece of code in a library (SPM) can build just fine in one project but fail to build in another project due to concurrency settings. The amount of overhead makes this very much unusable in a production / high pressure environment.

cosmic_cheese 5 days ago | parent | prev [-]

One of the biggest hurdles is just getting all the iOS/macOS/etc APIs up to speed with the thread safety improvements. It won’t make refactoring all that application code any easier, but as things stand even if you’ve done that, you’re going to run into problems anywhere your code makes contact with UI code because there’s a lot of AppKit and UIKit that have yet to make the transition.

potato-peeler 5 days ago | parent | prev | next [-]

I am curious. Generally basic structures like map are not thread safe and care has to be taken while modifying it. This is pretty well documented in go spec. In your case in dropbox, what was essentially going on?

tsimionescu 5 days ago | parent | next [-]

I think the surprise here is that failing to synchronize writes leads to a SEGFAULT, not a panic or an error. This is the point GP was making, that Go is not fully memory safe in the presence of unsynchronized concurrent writes. By contrast, in Java or C#, unsynchronized writes will either throw an exception (if you're lucky and they get detected) or let the program continue with some unexpected values (possibly ones that violate some invariants). Getting a SEGFAULT can only happen if you're explicitly using native code, raw memory access APIs, or found a bug in the runtime.

elcritch 4 days ago | parent [-]

Segfault sounds better than running with inconsistent data.

tsimionescu 4 days ago | parent | next [-]

No, SEGFAULT means you're lucky and your corrupted memory caused you to access something the OS knew you can't access. But every SEGFAULT means that you have a memory safety violation, and so if you get unlucky, the exact same code that SEGFAULTED once will read oe write random objects in your memory (which might include code areas, GC data structures, etc).

Inconsistent data is pretty bad, but it's not as bad as memory corruption.

foldr 4 days ago | parent | prev | next [-]

The inconsistent data thing could happen in Go too. A segfault is not guaranteed, it’s just one of the more likely possibilities.

Someone 4 days ago | parent [-]

> A segfault is not guaranteed, it’s just one of the more likely possibilities.

Is it? It will depend on the code, but my gut feeling is that you typically would get a few (if not lot of) unnoticed non-segfaulting issues before you get the segfaulting one that tells you straight in your face that you have a problem.

foldr 4 days ago | parent [-]

It probably depends on how exactly the corruption happens. If you overwrite a pointer with an integer value, then the integer is statistically unlikely to correspond to a valid memory address. On the other hand, if you overwrite a pointer with a pointer, or an integer with an integer, all bets are off.

Someone 3 days ago | parent [-]

> If you overwrite a pointer with an integer value, then the integer is statistically unlikely to correspond to a valid memory address

On 64-bit systems, and even then, it depends on the system’s memory layout (I think most integer values in programs are < 2³²)

foldr 2 days ago | parent [-]

Right. It’s unlikely both because the 64-bit value space is huge and because on most systems pointers have some of the high bytes set whereas typical integer values don’t. IIRC this combination of factors is what makes conservative GCs like BoehmGC quite effective on 64-bit architectures.

qcnguy 4 days ago | parent | prev | next [-]

Go/C programs that race can also run with inconsistent data. Nothing guarantees you a segfault under torn writes.

In practice, Java programs tend to pick up on data races very quickly because they mutate some collection and the collections framework has safety checks for this.

to11mtm 4 days ago | parent | prev [-]

Well it depends on what we mean by 'inconsistent'.

In C# For example, if a structure is over CPU arch Word size (i.e. 32 or 64 bits) then you could have a torn read if it's being written. However object refs themselves are always word size so you'll never have a torn pointer read on those.

However, in either case there is still a need in multithreaded environments to remember the CPU's memory ordering rules and put proper fences (or, to be safe, locks, since memory barrier rules are different between ARM and x86 for example).

But that second bit is a fairly hard problem to solve for without having the right type of modelling around your compiler.

tsimionescu 4 days ago | parent [-]

When I said "inconsistent", I was referring to things like getting a length field updated by one thread, but the actual list contents by another - if you have thread safety violations you will end up with exactly this type of issue in any language that allows unsafe threading code (Rust wouldn't outside `unsafe` blocks, for example), even in fully memory safe ones like Java or C#, and even without any bugs in the VM.

maxlybbert 5 days ago | parent | prev | next [-]

I thought the same thing. Maybe the point of the story isn’t “we were surprised to learn you had to synchronize access” but instead “we all thought we were careful, but each of us made this mistake no matter how careful we tried to be.”

nine_k 5 days ago | parent | prev [-]

In Java, there are separate synchronized collections, because acquiring a lock takes time. Normally one uses thread-unsafe collections. Java also gives a very ergonomic way to run any fragment under a lock (the `synchronized` operator).

Rust avoids all this entirely, by using its type system.

layer8 4 days ago | parent | next [-]

Java has separate synchronized collections only because that was initially the default, until people realized that it doesn’t help for the common cases of check-and-modify operations or of having consistency invariants with state outside a single collections (besides the performance impact). In practice, synchronized collections are rarely useful, and instead accesses are synchronized externally.

noisem4ker 5 days ago | parent | prev [-]

Golang has a synchronized map:

https://pkg.go.dev/sync#Map

CJefferson 4 days ago | parent | prev | next [-]

Before Rust, I'd reached the personal conclusion that large-scale thread-safe software was almost impossible -- certainly it required the highest levels of software engineering. Multi-process code was a much more reasonable option for mere mortals.

Rust on the other hand solves that. There is code you can't write easily in Rust, but just yesterday I took a rust iteration, changed 'iter()' to 'par_iter()', and given it compiled I had high confidence it was going to work (which it did).

rowanG077 4 days ago | parent | prev | next [-]

I'm pretty surprised by some other comments in this thread saying this is a rare occurrence in go. In my experience it's not rare at all.

Thaxll 4 days ago | parent | prev | next [-]

I have a hard time believing that it's common to create SEGFAULT in Go, I worked with the language for a very long time and don't remember a single time where I've seen that. ( and i've seen many data race )

Not synchronizing writes on most data structure does not create a SEGFAULT, you have to be in a very specific condition to create one, those conditions are extremely rares and un-usual ( from the programmer perspective).

In OP blog to triggers one he's doing one of those condition in an infinite loop.

https://research.swtch.com/gorace

commandersaki 4 days ago | parent [-]

You really have to go hunting for a segfault in Go. The critical sentence in OP article is: in practice, of course, safety is not binary, it is a spectrum, and on that spectrum Go is much closer to a typical safe language than to C. OP just has a vested interest in proving safety of languages and is making a big deal where in practice there is none. People are not making loads of unsafe programs in Go nor deploying as such because it would be pretty quickly detected. This is much different to C and C++.

commandersaki 5 days ago | parent | prev | next [-]

To put things in perspective, I posit to you, how many memory unsafe things can you do in Go that isn’t a variant of the same thing?

Or put another way what is the likelihood that a go program is memory unsafe?

tapirl 4 days ago | parent | prev | next [-]

Listens your team had not sufficient review capacity at that time.

5 days ago | parent | prev | next [-]
[deleted]
pjmlp 5 days ago | parent | prev | next [-]

It is kind of wild that for a 21st century programming language, the amount of stuff in Go that should have been but never was, but hey Docker and Kubernetes.

9rx 5 days ago | parent [-]

On the flip side, what would be the point? There are already a million other languages that have everything and the kitchen sink.

Not going down the same road is the only reason it didn't end up on the pile of obscure languages nobody uses.

pjmlp 5 days ago | parent | next [-]

The only reason it didn't end on pile of obscure languages nobody uses, it called Google, followed by luck with Docker and Kubernetes adoption on the market, after they decided to rewrite from Python and Java respectively into Go, after Go heads joined their teams.

Case in point, Limbo and Oberon-2, the languages that influenced its design, and authors were involved with.

tsimionescu 5 days ago | parent | next [-]

I don't think that's the (only) reason Go became popular. The huge thing about Go is the runtime: it's the only language runtime available today, at least in any language with a large org behind it, that offers (a) GC, (b) fast start-up time, (c) static types, (d) fast execution, and (e) multi-threading.

This is a killer combination for any team looking to write code for auto-scalable microservices, to run for example on Kubernetes. Java is not great in this niche because of its slow startup time, relatively large memory overhead, and the need for warm-up before code actually starts executing fast (so scaling up and down has a very large cost for Java services). .NET has similar problems, and also a huge container size. Python is far too slow, and not typed. TypeScript is single threaded, and still has a pretty hefty runtime. OCaml doesn't have any large org behind it, is quite obscure syntax, and was still single-threaded at the time Kubernetes started. Haskell has similar issues, and is also large and slow starting. Rust, C++, C all require manual memory management.

So, it's no surprise that Go was used for Kubernetes services themselves, and it's no surprise that people designing for Kubernetes mostly chose to write their new stuff in Go. Go the language, with its antiquated design, is actually quite secondary to all of that. But Go's runtime is completely unmatched in this space.

dev_l1x_be 4 days ago | parent | next [-]

F# does that too.

> .NET has similar problems

s/has/had/

https://blog.washi.dev/posts/tinysharp/

The issue is that some people still fighting against the concepts ML family languages (primarily SML) introduced. Go implemented go routines and channels from CSP (https://en.wikipedia.org/wiki/Communicating_sequential_proce...) but dragged a lot on influence from C (understandable) into the language.

I think Rust opted for the best combinations (some CSP, a lot of ML and a bit of C++).

9rx 3 days ago | parent | next [-]

> The issue is that some people still fighting against the concepts ML family languages

To be fair, everyone was fighting against ML concepts at the time. Ruby on Rails was "in" and "doing situps" was "out". Go was built for the time it was created. It was, quite explicitly as told at its launch announcement, made to be a "dynamically typed" language with static type performance. It is unlikely it would have had a static type system at all if they knew how to achieve the same performance optimizations without a type system.

> I think Rust opted for the best combinations

But built in another time. Ruby on Rails was "out" and static typing (ML style in particular) was "in" by the time Rust finally got around to showing up to the party. Looking back, it may not seem like there was much time between the creation of Go and the creation of Rust, but on the tech scale it was created eons later. The fashion of tech can change on a dime — as captured in the humorous fable about JS having a new "must-use" framework every week.

The fashion trends will change again at some point. They always do.

tsimionescu 4 days ago | parent | prev | next [-]

Not sure what you mean about F# - being a CLR language, it has the same runtime issues as C# (and IronPython, managed C++, etc).

The article you quote is a toy example - if you write a C# or F# web API server, you'll see that it takes up way more space than a Go one with similar functionality (and has way higher memory overhead as well). A Go API web server is maybe 10MB on disk, with no dependencies (that is, you can run it perfectly in a container that is defined as `FROM scratch; COPY my-go-exec /my-go-exec `). The equivalent Java or .NET container is somewhere around 2-400MB at the minimum.

As for the syntax and constructs, I don't care so much. If OCaml or SML had comparable support and a comparable ecosystem to Go, I'd bet plenty of people would have chosen them instead.

atombender 4 days ago | parent | prev [-]

OP's point is about why Go took off, which happened more than a decade ago.

Native AOT compilation didn't exist for .NET then, and .NET was, for all intents and purposes, Windows-only.

(Last I checked, the AOT story for C# is not that great even today. No idea about F#.)

qcnguy 4 days ago | parent | prev [-]

Kubernetes services are one of the places where you don't care about startup time. Likewise for Docker itself. These are the things that do the scaling, normally.

Go is not particularly fast. People often see that Java gets faster as it runs and thinks, oh, it must be slow at the start then. But when you compare like with like, Go ends up being stuck at the bottom of the curve that Java then sharply climbs. The difference in GC quality is big, or at least, used to be? The only place where you really notice the difference is command line tools, and Java has GraalVM for that.

tsimionescu 4 days ago | parent | next [-]

> Kubernetes services are one of the places where you don't care about startup time.

There are some kubernetes services that scale up and down. And even for those that don't normally, if they have some kind of failure, the difference between taking a millisecond to get back up and taking a second can actually matter for a web host.

> Go is not particularly fast. People often see that Java gets faster as it runs and thinks, oh, it must be slow at the start then. But when you compare like with like, Go ends up being stuck at the bottom of the curve that Java then sharply climbs.

Go starts up much faster than Java. And Go code runs measurably faster than interpreted Java code, even though it's slower than the JITed code you'll eventually have if your JVM runs long enpigh. But un-JITed Java code is very slow, more comparable to Python than JITed Java or with Go . This has nothing to do with the GC - where I do agree Go is mediocre at best.

zozbot234 4 days ago | parent [-]

I wouldn't call the Go GC mediocre, it's one of the few fully concurrent GC's in common use. It probably has significantly lower memory demand than Java/NET for comparable workloads.

tsimionescu 3 days ago | parent | next [-]

Go's GC is not compacting, it has significant mutator overhead on every pointer write, it has an expensive allocator (it has to search through free lists, since it can't compact), and the free pahse has to do work proportional to the amount of garbage the program produces. It also produces no debugging information for ownership, for some bizarre reason (you can't tell from a Go heap dump with any standard Go tool which object is keeping another object around).

The fact that it can do this mostly concurrently (it's not fully concurrent, actually) is not that special - Java's latest GC is also mostly concurrent, without having any of the other previous drawbacks.

Go's big advantage in terms of memory management is that it doesn't produce as much garbage as most GC languages. The fact that it natively supports value types, and actually uses them extensively in the standard library (unlike C#, which also supports these), and it's excellent stack escape analysis which allows many allocations to stay on the stack and outside the GC heap entirely, is what makes Go have such little memory overhead.

pebal 4 days ago | parent | prev [-]

This isn't fully concurrent GC. It pauses mutators threads and delegates them to perform some of the work for the GC.

atombender 4 days ago | parent | prev [-]

When using HPAs to quickly ramp replica sets up and down based on load, startup time is absolutely an important factor. You want services to start within hundreds of milliseconds at most.

Animats 5 days ago | parent | prev | next [-]

The strength of Go is not the language. It's that the libraries you need for web back-end stuff are written, maintained, and used in production by Google. All the obscure cases get exercised in production due to sheer volume of internal usage.

At one time, Go maps were not thread-safe. Was that fixed?

Yoric 5 days ago | parent | next [-]

I'd be surprised if the JSON module was used within Google, though. It's neither particularly fast nor particularly convenient nor particularly suited to properly handle edge cases. But it's still in the stdlib for compatibility reasons.

5 days ago | parent [-]
[deleted]
9rx 5 days ago | parent | prev | next [-]

> At one time, Go maps were not thread-safe. Was that fixed?

sync.Map was added, but isn't intended to be a general purpose map.

——

The Map type is specialized. Most code should use a plain Go map instead, with separate locking or coordination, for better type safety and to make it easier to maintain other invariants along with the map content.

The Map type is optimized for two common use cases: (1) when the entry for a given key is only ever written once but read many times, as in caches that only grow, or (2) when multiple goroutines read, write, and overwrite entries for disjoint sets of keys. In these two cases, use of a Map may significantly reduce lock contention compared to a Go map paired with a separate Mutex or RWMutex.

pjmlp 5 days ago | parent | prev [-]

Exactly, by Google.

9rx 5 days ago | parent | prev [-]

> The only reason it didn't end on pile of obscure languages nobody uses, it called Google

Dart ended up on the pile of languages nobody uses. And Carbon? What's Carbon? Exactly!

> Case in point, Limbo and Oberon-2, the languages that influenced its design

Agreed. Limbo and Oberon-2, as primitive as they may look now, had the kitchen sinks of their time. Why wouldn't they have ended up on the pile of languages nobody uses?

pjmlp 5 days ago | parent [-]

People love to bring those as counter examples, without actually knowing a single fact about them.

Dart was a victim of internal politics between the Chrome team, Dart team, AdWords moving away from GWT wanting AngularDart (see Angular documentary), and the Web in general.

Had Chrome team kept pushing DartVM, it might have been quite different story.

Carbon, good example of failure to actually know what the team purposes are. It is officially a research project for Google themselves, where the team is the first to advise using Rust or another MSL.

One just needs to actually spend like a couple of minutes on their wiki, but I guess that is asking too much on modern times.

Limbo and Oberon-2 were definitely not kitchen sinks of their time, their failure was that neither Bell Labs in 1996, nor ETHZ in 1992, were that relevant for the programming language community in the industry.

9rx 5 days ago | parent [-]

> Had Chrome team kept pushing DartVM, it might have been quite different story.

Trouble with that line of thinking is that Google never pushed Go either. It didn't even bother to use it internally (outside from the occasional side project here and there). Google paid some salaries. I'll give you that. But it has paid salaries for a lot of different languages. That is not some kind of secret sauce.

> It is officially a research project for Google themselves

It's not just a research project. It is officially "not ready for use", but its roadmap has a clear "ready for use" plan in the coming months. Rust was also "not ready for use" when it hit the streets, it officially being a Mozilla research project, but every second discussion on HN was about it and what is to come. And that was without Google backing. If what you say is true, why isn't Carbon being shouted from every rooftop right now?

I know you're struggling to grasp at straws here, but let's just be honest for a moment: If it hasn't caught attention already, it isn't going to. Just another language to add to the pile.

pjmlp 5 days ago | parent [-]

I guess Kubernetes doesn't count as Google pushing it then.

It is officially "not ready to use", it isn't a strawman as people keep complaining about nothing.

9rx 4 days ago | parent [-]

Kubernetes is written in Go (-ish. More like written in Java using Go syntax, especially in its early days when you claim it did some kind of pushing). But what kind of push are you dreaming up that it offered? If anything, Kubernetes leaves you to question if you ever want to write software again. The idea that it made anyone, let alone legions of people required to make something a "success", think "This is great. I should write my program in Go too!" is laughable.

stouset 5 days ago | parent | prev | next [-]

Having the weight of Google behind it is the primary reason it didn't end up on the pile of obscure languages nobody uses.

9rx 5 days ago | parent | next [-]

Oh...? Dart never gained much steam. And let's not forget about Carbon! Can you name even just one person who has tried Carbon? Have more than a handful of people even heard of Carbon?

I will grant you that Carbon is still in its infancy, but when Rust was in the same youthful stage we never heard an end to all the people playing with it. You, even if not tried it yourself, definitely knew about it.

You've made up a fun idea, but reality doesn't support it. Google has not shown its weight carries anything. They have really struggled to get any for-profit business units off the ground since they gained the weight, never mind their hobbies! If anything, Google is detrimental to a project.

pjmlp 5 days ago | parent | next [-]

Already explained in another thread, learn the politics of Dart, and Carbon is still on the drawing board.

geodel 5 days ago | parent | next [-]

So failures are some deep valid reasons whereas success is developers don't know any better language.

pjmlp 5 days ago | parent [-]

Many don't, otherwise they would not do stuff like using Python for performance workloads.

Lets also not forget Rob Pike famous quote regarding simple minds, as target audience.

As for Go, Kubernetes made it unavoidable, it is like UNIX for C, Web for JavaScript, and so forth.

9rx 4 days ago | parent [-]

> Many don't, otherwise they would not do stuff like using Python for performance workloads.

While you fairly point out that many fall into Python because they learned about it in school and never bother to look beyond, Go has had no such equivalent. For you to choose it, you have to actively seek it out, where you are going to also see all the other programming languages you could also choose.

> As for Go, Kubernetes made it unavoidable, it is like UNIX for C, Web for JavaScript, and so forth.

UNIX's programming interface is a set of C functions. You are right that C is the path of least resistance to use it.

The web's programming interface is Javascript. You are right that Javascript is the path of least resistance to use it.

Kubernetes' programming interface is a "REST API" – or your program running inside a container, if you want to look at it from the other direction. In what way is Go the path of least resistance to use it?

9rx 5 days ago | parent | prev [-]

Already...? Said "explanation" was posted over an hour after the comment replied to here.

If only Google put their weight into a watch, maybe you'd have one?

Oh wait. They did! Google can't successfully turn their weight into much of anything. Go's success, if we can call it that, clearly happened in spite of Google.

stouset 5 days ago | parent [-]

Googlers aren’t expected to wear a Google-branded watch at work. They are expected to write go. Having an entire Google’s worth of programmers using your programming language isn’t exactly a minor influence.

9rx 5 days ago | parent | next [-]

> They are expected to write go.

Like who? Outside of Go itself, which is really more of a community project — albeit with the chief maintainers still on Google's payroll, almost nothing at Google is written in Go. In fact, Pike once gave a talk reflecting on why it didn't succeed in that space, and noted that it was the "Python and Ruby programmers" who actually ended up adopting it.

Google makes money selling services (i.e. Google Cloud) that run Kubernetes, Docker, etc. If it weren't for that, it is unlikely that Google would even be continuing to maintain it at this point. It was an interesting experiment, perhaps, but ultimately a failure within Google. As before, it was the Python and (probably most especially) Ruby communities that ended up leaning into it.

Which isn't surprising in hindsight. Go offered those who were using Python and Ruby a language that was in the same kind of vein, while solving many of the pain points they were experiencing with Python and Ruby (awful deployment strategies, terrible concurrency stories, trouble with performance, etc.) These developers were never going to use Haskell. They wanted Ruby with less problems.

And that's what Go gave them — at least to the extent of being better than any other attempt to do the same. Since it solved real problems people had, without forcing them into new programming paradigms, it was adopted. Choosing a technology based on starry-eyed fandom and arbitrary feelings might be how you go about navigating this world, but that doesn't extrapolate.

geodel 5 days ago | parent | prev [-]

> They are expected to write go.

This got to be a joke right. The only thing I hear is at Google no one likes Go. Most software is in C++, Rust, Java or Kotlin.

stouset 5 days ago | parent | prev [-]

Necessary and sufficient are two separate concepts.

5 days ago | parent | prev [-]
[deleted]
Yoric 5 days ago | parent | prev [-]

Well, that and the slight fact that it bears Google's brand name.

I personally appreciate Go as a research experiment. Plenty of very interesting ideas, just as, for instance, Haskell. I don't particularly like it as a development language, but I can understand why some people do.

9rx 5 days ago | parent [-]

> Plenty of very interesting ideas

Is there? When you get down to it, it is really just a faster Python. Which is exactly what it was said to be when it was released. Their goal was to create a "dynamically-typed" language that was more performant. It is likely that it wouldn't have had a static type system at all if they figured out how to achieve on the performance end without needing types.

You can tell who is clueless when you hear someone say its type system is lacking. I mean, technically it is, but it is supposed to be. Like saying Javascript or Ruby's type system is lacking.

pjmlp 5 days ago | parent | next [-]

Faster Python with a very small set of its capabilities.

Yoric 5 days ago | parent | prev [-]

Two examples of interesting ideas:

- using zero values as an optimization mechanism;

- (non-)pointers and passing self by copy.

I mean, I hate both mechanisms, but intellectually, I find them quite interesting.

Also, I'd not classify it as a faster Python. It's more of a cousin of Obj-C if the authors of Obj-C had fallen in love of Erlang instead of Smalltalk.

bombela 4 days ago | parent [-]

Those ideas where already present in C/C++ decades prior. BSS program area. And passing struct by value vs pointer.

nosefrog 4 days ago | parent | prev | next [-]

Hi Chad!

gok 5 days ago | parent | prev | next [-]

Java is not memory-safe in the Rust sense.

ferreiratb 5 days ago | parent [-]

Can you elaborate on that?

wpollock 5 days ago | parent [-]

A Java program can share mutable state between threads without synchronization, and it will compile and run. In Rust, such a program will not compile.

int_19h 5 days ago | parent [-]

Yes, but even so you will never see e.g. an invalid pointer value as the result of a torn memory write. Basically, no matter what you do with threads in Java, it will not segfault.

TFA's point is that (safe) Rust is also like that, but achieves it by restricting all cases where a torn write could be observed through its type system instead of VM's memory model.

dontlaugh 4 days ago | parent [-]

More specifically, Rust prevents data races.

BlackFly 2 days ago | parent [-]

No, rust forces you to use a mutex but nothing will prevent you from making the mutex too small and creating tearing in your own data structures by sequentially modifying things covered by mutexes so that in between acquisition of the locks you are violating invariants. The borrow checker certainly helps however, but not without cost that was finally minimized when the scoped threads api came along.

Java has a very specific memory model, so the behavior of variables across threads is quite well defined. Basic variables can tear however (a 64bit long on a 32bit architecture) without the volatile keyword and that is quite different than rust.

dontlaugh 2 days ago | parent [-]

You didn’t describe any data races.

What Rust prevents is very specific.

int_19h a day ago | parent [-]

OP described situations where you get observable invariant violations because of torn non-atomic writes. This is basically any case involving e.g. copying of variables that are larger than whatever's atomic for a given architecture. Say, a struct of 4 isize.

adamwk 5 days ago | parent | prev | next [-]

Crashing on shared access is the safe thing to do

mirashii 5 days ago | parent | next [-]

An intentional exit by a runtime is a safe crash. A segfault is not, and is here a clear sign that memory safety has been violated.

Gibbon1 5 days ago | parent | next [-]

Yeah it's not the segfault that's bad, it's when it's when the write to address 0x20001854 succeeds and now some hapless postal clerk is going to jail.

adamwk 5 days ago | parent | prev [-]

I guess I was thinking specifically of the swift case where values have exclusive access enforcement. Normally caught by a compiler, they will safely crash if the compiler didn’t catch it. I think the only way to segfault would be by using Unsafe*Pointer types, which are explicitly marked unsafe

sapiogram 5 days ago | parent | prev | next [-]

"Crashing" is a very positive spin. "The heap getting the corrupted until it was killed by the operating system" is another interpretation.

LtWorf 5 days ago | parent | prev [-]

a segfault is completely unintentional. Had the kernel been older it could be used to execute code.

zbentley 5 days ago | parent [-]

> a segfault is completely unintentional

Usually, but not always! https://jcdav.is/2015/10/06/SIGSEGV-as-control-flow/

gowld 4 days ago | parent [-]

> Faulted trying to access 0x10 - the offset in the string we were trying to read from :)

Is guaranteed that every offset you can try to read is guaranteed to create a segfault?

cesarb 4 days ago | parent [-]

> Is guaranteed that every offset you can try to read is guaranteed to create a segfault?

The offset is fixed as part of the compiled code; the JVM can enforce that it's less than 4k (otherwise it can use an explicit NULL check), and that the first 4k page is always unmapped.

Mawr 5 days ago | parent | prev | next [-]

Safety isn't binary, so your comment makes no sense.

kstrauser 5 days ago | parent | next [-]

I’d argue that unsafety is binary. If a normal eng doing normal things can break it without going out of their way to deliberately fool the compiler or runtime, I’d call it unsafe.

Calavar 5 days ago | parent [-]

By that definition Rust also counts as unsafe. Even managed languages like C# and Java would be unsafe.

gpm 5 days ago | parent | next [-]

There's a reason why rust devs qualify it as "memory safe" so frequently, we tend to agree that rust is, like virtually every current programming language, unsafe in other ways.

Memory safety is just the source of bugs that we've figured out how to eliminate. It's a significant source of really bad (hard to debug due to action at a distance, high impact, etc) bugs so that's worth a lot, but it's not perfect. And even then we have a more frequently used escape hatch to the memory-unsafe world than would be ideal from a safety perspective for practical reasons.

A more complete version of safety would be achieved with a language that proves code correct to arbitrary specifications. We aren't there yet for there being such a language that is practical for every day use. Personally I'm increasingly optimistic we'll get there sooner rather than later (say, within 20 years). Even then there will probably be specification level bugs that prevent a claim of complete safety...

kstrauser 5 days ago | parent | prev | next [-]

My impression of the Rust devs is that they’d agree with you about any easy-to-trigger calamities. So would Java contributors. C# might not because MS is institutionally not good about admitting mistakes, but I bet the individual devs would agree over a beer.

ackfoobar 5 days ago | parent | prev | next [-]

Do you have some examples? I think JDK developers make a lot of effort to make sure users bugs will not corrupt the runtime.

dcminter 5 days ago | parent | prev [-]

What kinds of breakage do you have in mind though? The number of times I've segfaulted the JVM is tiny.

sapiogram 5 days ago | parent | prev [-]

"Memory safety" has a specific, binary definition.

kiitos 2 days ago | parent | next [-]

this discussion should be ample evidence that it very clearly does not

0xbadcafebee 5 days ago | parent | prev [-]

It has two definitions?

shadowgovt 5 days ago | parent | prev [-]

Mostly because it was a remarkable improvement over what came before (and what came before was hilariously fragile).

int_19h 5 days ago | parent | next [-]

It was certainly not a remarkable improvement in the sense of being memory safe even in the face of race conditions. As the article points out, Java and C# both managed to do that, and both predate Go.

pjmlp 5 days ago | parent | prev [-]

Only for those not paying attention outside mainstream, or too young to remember former languages.

shadowgovt 5 days ago | parent | next [-]

I'm certainly not disagreeing, but I will note that by definition, most people are in the mainstream, so something being a remarkable improvement over what came before (in the mainstream) is a remarkable improvement (for most people).

the_plus_one 5 days ago | parent | prev [-]

> or too young to remember former languages.

Do you have any good examples? Not trying to argue, just genuinely curious as someone who hasn't been in this field for decades.

LtWorf 5 days ago | parent | next [-]

Basically go was designed ignoring all the research and progress that had been made in programming languages until then.

It was designed with contempt for developers, for example disallowing developers to create generic data structures, or lacking a decent way of error checking that is not extremely error prone and verbose.

5 days ago | parent | prev [-]
[deleted]