> Of course some engineers need to implement hash tables, or sorting algorithms or whatever.

> We love those engineers: they write libraries we can use off the shelf so we don’t have to implement them ourselves.

The world needs to love "infrastructure developers" more. To me it seems only the killer app writing crowd is valued. Nobody really thinks about the work that goes into programming languages, libraries and tools. It's invisible work, taken for granted, often open source, not rarely unpaid.

> It wasn’t opening a textbook to find the famous algorithm that would solve my problem.

I had that exact experience. I'm working on my own programming language. After weeks of trying to figure something out by myself, someone told me to read Structure and Interpretation of Computer Programs. It literally had the exact algorithm I wanted.

▲

chamomeal 4 days ago | parent [-]

Dang I got this book a few weeks ago and still haven’t cracked it open. Maybe today is the day

▲

matheusmoreira 4 days ago | parent | next [-]

The algorithm I'm talking about is at the very end of the book. If you start reading it from start to finish you might stop before you reach it. Certainly happened to me. Someone had to point it out for me to realize SICP had the answer all along.

https://eng.libretexts.org/Bookshelves/Computer_Science/Prog...

The explicit control evaluator. It's a register and stack machine which evaluates lisp expressions without transforming them into bytecode.

▲

hinkley 4 days ago | parent [-]

A common story with JIT languages is to go back and forth between having a bytecode interpreter and not.

The paradox is that when the interpreter is fast enough then you delay JIT because it takes longer for the amortized cost to be justified. But that also means the reasons for that high amortization cost don’t get prioritized because they don’t really show up as a priority.

Eventually the evidence piles so high nobody can ignore it and the code gets rearranged to create a clearer task list. And when that list runs out they rearrange again because now that other part is 2x too slow.

Personally I’d love to see a JIT that was less just in time. Queuing functions for optimization that only get worked on when there are idle processors. So there’s a threshold where the JIT pre-empts, and one where it only offers best effort.

▲

matheusmoreira 4 days ago | parent [-]

I wanted to preserve the "code is just lists" property of lisps. Compiling the lists away means that property is lost: the code becomes bytecode or native code instead, sacrificing lisp's soul in exchange for performance.

I want to implement a partial evaluator one day. That should go a long way to improving performance by precomputing and inlining things as much as possible.

	▲	hinkley 4 days ago \| parent [-]
		Sometimes people avoid that by making a stupidly cheap code generator that goes straight from the input file format to unoptimized machine code. Because you only have to reach a fraction of what the optimized code would achieve for throughput.

▲

alabhyajindal 4 days ago | parent | prev [-]

This might be useful: https://docs.racket-lang.org/sicp-manual/index.html