I only use the one model that I'm provided for free at work. I expect that's most users behavior. They stick to the one they pay for.

Best I can do is give you one that failed on GPT-4o

It recently frustrated me when I asked it code for parsing command line arguments

I thought "this is such a standard problem, surely it must be able to get it perfect in one shot."

> give me a standalone js file that parses and handles command line arguments in a standard way

> It must be able to parse such an example

> ```

> node script.js --name=John --age 30 -v (or --verbose) reading hiking coding

> ```

It produced code that:

* doesn't coalesce -v to --verbose - (i.e., the output is different for `node script.js -v` and `node script.js --verbose`)

* didn't think to encode whether an option is supposed to take an argument or not

* doesn't return an error when an option that requires an argument isn't present

* didn't account for the presence of a '--' to end the arguments

* allows -verbose and --v (instead of either -v or --verbose)

* Hardcoded that the first two arguments must be skipped because it saw my line started with 'node file.js' and assumed this was always going to be present

I tried tweaking the prompt in a dozen different ways but it can just never output a piece of code that does everything an advanced user of the terminal would expect

Must succeed: `node --enable-tracing script.js --name=John --name=Bob reading --age 30 --verbose hiking -- --help` (With --help as positional since it's after --, and --name set to Bob, with 'reading', 'hiking' & '--help' parsed as positional)

Must succeed: `node script.js -verbose` (but -verbose needs to be parsed as positional)

Must fail: `node script.js --name` (--name expects an argument)

Should fail: `node script.js --verbose=John` (--verbose doesn't expect an argument)

▲

alex_duf 8 months ago | parent [-]

Have you tried claude?

https://claude.ai/public/artifacts/9c2d8d0c-0410-4971-a19a-f...

node script.js --name=John --age 30 -v

Parsed options: { name: 'John', age: 30, verbose: true, help: false }

Positional arguments: []

node script.js --name=Alex --age 40 -v

Parsed options: { name: 'Alex', age: 40, verbose: true, help: false }

Positional arguments: []

▲

sebstefan 8 months ago | parent | next [-]

I keep seeing that `args = process.argv.slice(2)` line to skip past `node script.js`

I ended up settling for it as well (I couldn't find anything better, nor make it break) but I'd be really surprised if it was the way to go

Like `node --enable-tracing script.js --name=John --age 30 --verbose`

This works because node seems to hide --enable-tracing to the underlying script

But would it work with Bun & Deno...? Is that standard...?

▲

sebstefan 8 months ago | parent | prev [-]

This one seems way better

It didn't account for the presence of a '--' to end the parsing of named arguments but that's it

▲

echoangle 8 months ago | parent [-]

> It didn't account for the presence of a '--' to end the parsing of named arguments but that's it

That’s just something getopt does and some programs adopted. If you asked me to write a parser, I wouldn’t necessarily include that either if you didn’t ask for it.

▲

sebstefan 8 months ago | parent [-]

If you don't include it you can't have positional arguments that look like options

Some positional arguments can be filenames, filenames can be --help and --verbose or --name=Frank

You have to have `--` or something similar to have a correct program

▲

echoangle 8 months ago | parent [-]

> You have to have `--` or something similar to have a correct program

No, only if the positional arguments need to support arbitrary strings. If you have something like a package manager and the first positional argument is the subcommand and everything after is an alphanumeric package name, you don’t need to support the double dash.

	▲	sebstefan 8 months ago \| parent \| next [-]
		But also like as a matter of principle it's annoying when you're working on a big codebase, and you have weird, hidden, silently explosive unwritten contracts between random components Package managers is an especially bad example because github projects typically are github.com/author-name/words-separated-by-dashes So you will probably have somebody along the way pester you about allowing dashes between words, to play nice with github But who's to know if the guy making that change will think of disallowing dashes at the start of the words? Likely he'll just add \- to /[A-Z\-]+/ Suddenly the script you wrote starts getting passed positional arguments that have dashes in them, until some wise guy tries to create a package called `--verbose`, then notices unintended effects on your pages, goes ahead trying `--verbose -- react`, ... So anyway -- if I'm making a heavily reusable piece of code like this I make it as general purpose as possible in a way that makes it impossible to mis-use it
	▲	sebstefan 8 months ago \| parent \| prev [-]
		You don't know what it's going to be used for when you get that prompt. Not handling arbitrary strings is a bug