Our backend is primarily in Scala and has been for over a decade. My take is that Scala 2 -> 3 was a massive hit to the language's momentum. It came at a time when the language's popularity was waning and it had all the same challenges that other major language version changes tend to have. Migrating's been quite challenging. Tooling, which was already a challenge for the language, got worse because it was split between Scala 2 and 3.

The language was deliberate about providing backwards and forwards compatibility between 2 and 3, but they didn't have a good story for macros. Yes, macros were experimental, but the ecosystem relied on them very heavily. This meant that your 3rd party dependencies likely hadn't migrated to Scala 3. If they had, things got complicated because you had to deal with two macro systems.

After many years the tooling is finally catching up and many 3rd party dependencies are available in Scala 3. That said, it's taken years and years. I think during those years many companies migrated to other languages or put a moratorium on new projects in Scala.

Not only it happened at the worst possible time, but some pain points are entirely self-inflicted, like the braceless syntax hurting tooling and fragmenting the community for very little reason.

But in my opinion Databricks implicitly saying "no thanks" for the foreseeable future did a lot of damage here. Databricks didn't even bother bringing their proprietary runtime to Scala 2.13 until last year. Scala 3 is not entirely out of the question but it's taking forever, and at this point users can seriously ask themselves if Spark will actually stay relevant that long.