I've more or less gotten used to the jeans.

Probably due to a combination of undiagnosed autism and some very mild bullying in middle school (due to me growing rapidly and people making fun of me for wearing "short shorts"), I have been almost exclusively wearing jeans for everywhere but the gym since eighth grade.

The hoodies are a more recent thing and are more miserable. I'll check out linen clothing.

Yeah, there are linen pants too, so you don't have to wear shorts if you don't feel comfortable showing your legs. Linen in general is a great option if you want to not die of heat and you don't feel like showing your whole body to the world.

Since you're autistic, you might want to buy them in person: the tactile sensation of linens can be pretty different, so you might want to touch them before buying if sensory issues are a clothing concern.

One thing to look out for is that a lot of clothing that will come up when you search for linen are linen blends, which are better than a hoodie by far, but aren't going to have the same breeziness as 100% linen. Also, 100% linen wrinkles, so you've got to hang it right out of the dryer if you care about that.