Scala 3 is sorta a new language, bringing a lot of improvements to the type system: https://docs.scala-lang.org/scala3/new-in-scala3.html

It also looks like it has some improvements for dealing with `null` from Java code. (When I last used it I rarely had to deal with null (mostly dealt with Nil, None, Nothing, and Unit) but I guess NPEs are still possible and the new system can help catch them.)

This is one example why being a guest language isn't optimal.

They should have made use of JVM bytecodes that allow to optimize lambdas away and make JIT aware of them, via invokedynamic and MethodHandle optimizations.

Naturally they cannot rely on them being there, because Kotlin also needs to target ART, JS runtimes, WebAssembly and its own native version.

The killer is specifically the inlining of macros -- which Kotlin lacks.

And not all macros, but just the ones which expand to massive expressions

Think template expressions in C++ or proc macros in Rust

> it's a language made by academics for academics to play with language design. It was a little weird it blew up in industry for a while.

Yep. They have always been pretty honest about this.

I think that it blew up in industry because it really was ahead of its time. Type systems were pretty uncool before Scala. It proved that you could get OO and FP in a single type system.

Actually, a big part of reason for doing Scala 3 was rebasing the language on a more rigorous basis for unifying OO and FP. They felt that for all their other big ideas, it was time to rethink the fundamentals.

>It's sad but I generally agree. Scala was in my view pretty well positioned for an up and coming language ~2010-15

I used Scala for a bit around that period. My main recollection of it is getting Java compiler errors because Scala constructs were being implemented with deeply nested inner classes and the generated symbol names were too long.

It would have been a transitive dependency based on the comments about the library being "transparent" and the author unaware it was even used.

Pinning specific versions of transitive deps is fairly common in large JVM projects due to either security reasons or ABI compatibility or bugs

I agree for measuring latency differences you want similar setups. However, by running two versions of the app concurrently on the same machine they both get impacted more or less the same by noisy neighbours. Moreover, by inspecting the flamegraph you can, manually, see these large shifts of time allocation quickly. For automatic comparison you can of course use the raw data.

In addition you can look at total cpu seconds used, memory allocation on kernel level, and specifically for the jvm at the GC metrics and allocation rate. If these numbers change significantly then you know you need to have a look.

We do run this benchmark comparison in most nightly builds and find regressions this way.

For production systems I use flight recordings (jfrs). To analyze I use java mission control.

For OOME problems I use a heap dump and eclipse memory analysis tool.

For microbenchmarks, I use JMH. But I tend to try and avoid doing those.

I use jmh for micro benchmarks on any code we know is sensitive and to highlight performance differences between different implementations. (Usually keep them around but not run on CI as an archive of what we tried.)

Then we do benchmarking of the whole Java app in the container running async-profiler into pyroscope. We created a test harness for this that spins up and mocks any dependencies based on api subscription data and contracts and simulates performance.

This whole mechanism is generalised and only requires teams that create individual apps to work with contract driven testing for the test harness to function. During and after a benchmark we also verify whether other non functionals still work as required, i.e. whether tracing is still linked to the right requests etc. This works for almost any language that we use.

jmh is what I've always used for small benchmarks.

async-profiler

Why is your personal experience relevant to the wider market? How many companies and locations did you survey for that?

another issue with kotlin, because it encourage Java ecosystem usage like Spring is not much differentiation that could drive adoption.

> My understanding is that Java added a lot of functional programming

This is true, but needs more context. Java 8 added Stream API, which (at this time) was a fantastic breath of fresh air. However, the whole thing felt overengineered at many points, aka - it made complex things possible (collector chaining is admittedly cool, parallel streams are useful for quick-and-dirty data processing), but simple everyday things cumbersome. I cannot emphasize how tiring it was to have to write this useless bolierplate

  customers.stream().map(c -> c.getName()).collect(Collectors.joining(", "))

  customers.map(c -> c.getName()).join(", ")

Sort of true, but I often hear this take from Java programmers and it feels like "Blub" [1]/Stockholm syndrome to me.

Personally, I'm extremely glad to not have had to write .toStream().map(...).collect(Collectors.list()) or whatever in years for what could be a map. Similar with async code and exception handling.

For me one of the main advantages of Kotlin is that is decreases verbosity so much that the interesting business logic is actually much easier to follow. Even if you disregard all the things it has Java doesn't the syntax is just so much better.

[1] https://paulgraham.com/avg.html

My org had to write a pivotal backend service on the JVM, due to JDBC having the largest number of data source adapters.

The choice was Kotlin. Scala is too "powerful" and can be written in a style that is difficult for others, and Java too verbose.

Kotlin is instantly familiar to modern TypeScript/Swift/Rust etc devs.

The only negative in my mind has been IntelliJ being the only decent IDE, but even this has changed recently with Jetbrains releasing `kotlin-lsp` for VS Code

https://github.com/Kotlin/kotlin-lsp

At least where I work, writing new Java code is discouraged and you should instead use Kotlin for backend services. Spring Boot which is the framework we use, supports Kotlin just fine, at the same level as Java. And if you use Jetbrains tools, Kotlin tooling is also pretty good (outside Jetbrains I will admit it is worse than Java). Now, even in new Java projects you can still be using Kotlin because it is the default language for Gradle (previously it was Groovy).

Java's new features are always going to be on paper. The ecosystem, with all its legacy code, is always going to be a decade behind. And if you are starting a new project, why would you pick Java over Kotlin?

When I checked a year ago, the IDE tooling still wasn't quite there.

Except the reason behind the syntax change is the losing mindshare from Scala into Python, after its relevance in the big data wave that predated the current AI wave.

Nothing to do with Haskell, even if it is also white space significant.

It's been a while since I touched Scala but wasn't that a thing in previous versions, minus the braces not being present?

Sorry, I'm coming from C++-ish background - can anyone explain what's going on :)

madness :)

That doesn’t follow. Scala is a high level language and compiler above the JVM. The bug here is a high level one:

> Turns out there was indeed a subtle bug making chained evaluations inefficient in Scala 3

I’m comparing with Haskell, Scheme, or even SQl which all promise to compile efficient code from high level descriptions.

That's not true, Spark's entire query engine relies on use of runtime codegen via macros/quasi quotes

Look up the architecture of Catalyst + Tungsten

https://www.databricks.com/glossary/catalyst-optimizer

Thank you for the clarification. If I understand correctly, these large expressions are created at compile-time, but the impact isn't felt until JIT occurs in the runtime environment. In that scenario, shouldn't the JIT just run once at startup, though? I'm still not quite understanding how JIT can take so much time in a production environment.

Many of the Scala projects got people fired. Something the Scala devs largely ignore. Plus Scala support is truly awful even by the low standards of an OpenSource project. Then there is the fact that the Scala specific libraries are largely dead.

Scala had/has a lot of promise. But how the language is marketed/managed/maintained really let a lot of people down and caused a lot of saltiness about it. And that is before we talk about the church of type-safety.

Scala is a more powerful language than Kotlin. But which do you want? A language with decent support that all your devs can use, or a language with more power but terrible support and only your very best devs can really take advantage of. And I say this as someone writing a compiler in Scala right now. Scala has its uses. But trying to get physicists used to Python to use it isn't one of them. Although that probably says more about the data science folks than Scala.

PS The GP is right, they should have focused on support and fixing the problems with the Scala compiler instead of changing the language. The original language spec is the best thing the Scala devs ever made.