I'm on Spark Scala 2 project and I hate it. Basically any good Scala dev would never want to work on our ETL projects, so we get second rate Python or Java devs like me who bastardize the language to get anything to work. Most of our new stuff is all pyspark, hopefully we can replace Scala asap.