Remix.run Logo
aleph_minus_one 6 hours ago

> Dynamically typed languages generally have a lower LLM token cost than traditional statically typed languages because omitting explicit type declarations makes the code more compact.

If this was true, the programming languages that are very much on the left side of

> https://danuker.go.ro/programming-languages.html#non-math-ma...

> https://danuker.go.ro/programming-languages.html#overall-map

should be very ideal for LLMs, in particular if they are dynamically typed.

What I can tell you is: I experimented with AI prompts for generating Wolfram (Mathematica) code using some LLMs, and I can tell you that the results were very disappointing: in my experience LLMs have difficulties with programming languages that are

- very concise, and

- for which there is less code publicly available.

Wolfram (Mathematica) is a good example of such a programming language.

acchow 2 hours ago | parent | next [-]

> omitting explicit type declarations makes the code more compact.

I guess this ignores languages with type inference? Hindley-Milner and others

JoeyJoJoJr 5 hours ago | parent | prev | next [-]

I’ve actually found Sol delivers great results with Odin, despite there not being much Odin code available. I think it is able to work well with it because:

- It is a rather simple language - It has a lot of very useful libraries already built in.

With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.

aleph_minus_one 4 hours ago | parent | next [-]

> I think it is able to work well with it because:

> - It is a rather simple language - It has a lot of very useful libraries already built in.

> With just a single main.odin file you can do a heck of a lot stuff, which LLMs seem to like.

Also Wolfram/Mathematica has an insane amount of useful libraries already built in (there even exists the saying "Python is 'batteries included', Wolfram is 'spaceship included'"), and also there in a single file you can do a heck of a lot stuff.

On the other hand:

- LLMs tend to hallucinate non-existing function when you ask an LLM to code something in Wolfram that is not commonly done (concerning this point, nevertheless keep in mind that Wolfram is often used for "one-of-a-kind programs", i.e. for writing very specialized programs that have possibly never been done before).

- Wolfram code tends to be quite dense.

- If there is a small mistake in Wolfram code, the code typically simply won't work.

petra 2 hours ago | parent [-]

Is there a way in Wolfram to check whether all function names exist ? And than give it as feedback to the llm?

aleph_minus_one an hour ago | parent [-]

> Is there a way in Wolfram to check whether all function names exist ?

There is a way to check whether a symbol has been defined:

  ValueQ[FunctionName, Method -> "SymbolDefinitionsPresent"]
See https://reference.wolfram.com/language/ref/ValueQ.html

Replace FunctionName by the function name that you want to check.

ch4s3 2 hours ago | parent | prev [-]

It’s interesting I’ve been surprised by how well Claude sonnet can write code in a language I’m developing that probably has no code in the training set. It seems like anything with syntax like python/ruby/elixir is pretty LLM friendly, and layering on a HM type system seems to help catch most errors.

frollogaston 2 hours ago | parent | prev [-]

Training data is a factor too