| ▲ | izabera 15 days ago |
| >sufficiently descriptive natural language specification
https://www.commitstrip.com/en/2016/08/25/a-very-comprehensi... |
|
| ▲ | intelVISA 15 days ago | parent | next [-] |
| sounds like it would pair well with a suitably smart compiler |
| |
| ▲ | soulofmischief 15 days ago | parent [-] | | I wrote one! It works well with cutting-edge LLMs. You feed it one or more source files that contain natural language, or stdin, and it produces a design spec, a README, and a test suite. Then it writes C code, compiles with cosmocc (for portability) and tests, in a loop, until everything is passing. All in one binary. It's been a great personal tool and I plan to open source it soon. |
|
|
| ▲ | soulofmischief 15 days ago | parent | prev [-] |
| No, the key difference is that an engineer becomes more product-oriented, and the technicalities of the implementation are deprioritized. It is a different paradigm, in the same way that a high-level language like JavaScript handles a lot of low-level stuff for me. |
| |
| ▲ | soraminazuki 15 days ago | parent [-] | | A programming language implementation produces results that are controllable, reproducible, and well-defined. An LLM has none of those properties, which makes the comparison moot. Having an LLM make up underspecified details willy-nilly, or worse, ignore clear instructions is very different from programming languages "handling a lot of low-level stuff." | | |
| ▲ | soulofmischief 15 days ago | parent [-] | | [citation needed] You can set temperature to 0 in many LLMs and get deterministic results (on the same hardware, given floating-point shenanigans). You can provide a well-defined spec and test suite. You can constrain and control the output. | | |
| ▲ | soraminazuki 15 days ago | parent [-] | | LLMs produce deterministic results? Now, that's a big [citation needed]. Where can I find the specs? Edit: This is assuming by "deterministic," you mean the same thing I said about programming language implementations being "controllable, reproducible, and well-defined." If you mean it produces random but same results for the same inputs, then you haven't made any meaningful points. | | |
| ▲ | soulofmischief 15 days ago | parent [-] | | I'd recommend learning how transformers work, and the concept of temperature. I don't think I need to cite information that is broadly and readily available, but here: https://medium.com/google-cloud/is-a-zero-temperature-determ... I also qualified the requirement of needing the same hardware, due to FP shenanigans. I could further clarify that you need the same stack (pytorch, tensorflow, etc) | | |
| ▲ | soraminazuki 15 days ago | parent [-] | | This gcc script that I created below is just as "deterministic" as an LLM. It produces the same result every time. Doesn't make it useful though. echo '#!/usr/bin/env bash' > gcc
echo 'cat <<EOF' >> gcc
openssl rand -base64 100 >> gcc
echo 'EOF' >> gcc
chmod +x gcc
Also, how transformers work is not a spec of the LLM that anyone can use to learn how LLM produces code. It's no gcc source code. |
|
|
|
|
|
