Remix.run Logo
▲ frollogaston 3 hours ago

  Also, in most languages an error will crash your program. So if you’re writing code with an LLM it will have to read your crash logs to make some changes and run your program again. In Common Lisp your program won’t crash, it’ll stop and open a debugger with the whole stack and all the variables. You can just point your LLM at the debugger, and it’ll make its fix and resume the program.
What does this look like in a real example system that you're maintaining? I can't imagine you'd always be able to resume like that if it's something like a webserver.
▲chrchr 2 hours ago | parent | next [-]

Test driven developmet practitioners in Smalltalk used to (still do?) write the test before writing the implementation, run the test, and, when the "method not implemented" error shows up in the debugger, type the implementation and continue the execution. Common Lisp can do that too.

So, in practice, it might look like you hit any kind of runtime failure, and then the LLM writes some code to fix it, and the user's request completes successfully with no errors.

▲rmunn 3 hours ago | parent | prev | next [-]

Not the author, but unless the server was running in a short-lived ephemeral container (in which case the management system probably killed the container and started a new one), the CL process would still be around in a paused state, waiting for you to connect to it and tell it how to resume.

https://comp-348.github.io/lisp-debugging.html has an example of what it looks like. A toy example, to be sure, but the basics of a real example would still look the same. You're given a choice of several options, very reminiscent of the "Abort, Retry, Ignore?" choice that used to be oh-so-familiar in the days of DOS. Except this one is more useful, because it offers ways to specify how to resume. E.g., the toy project is halting on a `(print X)` call where the value of X is not defined. And the choices are:

0. Continue. (Retry using X).

In the toy example, this would fail, because nothing else has defined X. But in real code, the name might have been undefined because the data needed to define it hadn't arrived yet, from the database or the filesystem. In which case retrying the statement might work the second time.

1. Use-value. (Use specified value).

This one prompts you to enter a value for the undefined variable, and continues, but it does not modify the value of X in the program. The next time the program tries to use X, it will halt again with another "unbound variable" error.

2. Store-value. (Set specified value and use it).

This one, just like Use-value, will prompt you to enter a value to use... but then it will set X to that value and continue running the program. Next time the program tries to read the value of X, it will have one, and the program won't halt.

3. Abort. (Exit debugger, returning to top level).

This is what you would choose if there's no good way to fix the error, and you just have to quit the program and restart. Though note that choosing this option isn't going to exit the program you're debugging, just take you out of the debugger. You'll still need to kill-and-restart it some other way... or come back an hour later when the value is finally available, and then choose options 1 or 2.

Hopefully that gives you a taste for what the CL debugger is like to use in practice.

▲frollogaston 3 hours ago | parent [-]

This looks like a good debugger, but it's similar to ones I've used in other languages. What's special about this one? I thought the idea was you use the debugger in prod, but now I don't think that's what the article meant.

▲rmunn 3 hours ago | parent [-]

Yes, you use the debugger in prod. I've read many articles by Common Lisp users talking about how great that feature is: they can quickly get production back up and running, then go to the code and implement the same fix they just did on live prod.

And it's not editing files on the server, it's actually reaching into the running code and tweaking its values.

That, I think, is the difference here. In many languages, the debugger can pause on the exception and let you inspect the code. But in every other language I've used, once you edit the code to fix the bug, you can't resume from where the debugger paused. You have to recompile the code and resume from the top. In CL, you can resume from exactly the state you were in when the debugger paused, only this time with the correct data in place. (Or even with a code fix having been applied, live, to the code).

▲darkwi11ow 3 hours ago | parent | prev [-]

You can do this in many other interpreted languages like Python interpreter when running with --pdb flag will do the same. I imagine CL does this by default when running code in interpreted mode and omits the behaviour for compiled (production) binaries.

▲frollogaston 3 hours ago | parent | next [-]

Yeah but what happens to your server then? The connection it was handling times out I guess, then is the rest of it frozen or what?

▲rmunn 3 hours ago | parent [-]

I mean, that entirely depends on the server design. Is it running one connection per process? One connection per thread? Presumably you've designed the server in a way that one slow connection doesn't block other connections from being handled fast, which means that this one connection whose handler went into the debugger is just now a really, really slow connection taking minutes (or hours!) to handle, but the other connections are running just fine.

Now, if your entire server is taken down because one connection threw an exception, that's bad design. But pretty much no major language works that way. All of them allow you to set things up so that an exception handling connection A won't affect connection B. And if you've done that in Common Lisp, then connection A halting and waiting for the debugger won't affect connection B either.

▲Jtsummers 3 hours ago | parent | prev [-]

In CL, this works with compiled code.