Remix.run Logo
johnisgood 3 days ago

I much prefer Claude over ChatGPT, based on my experience using both extensively. Claude understands me significantly better and seems to "know" my intentions with much greater ease. For example, when I request the full file, it provides it without any issues or unnecessary reiterations (ChatGPT fails after me repeatedly instructing it to), often confirming my request with a brief summary beforehand, but nothing more. Additionally, Claude frequently asks clarifying questions to better understand my goals, something I have noticed ChatGPT never did. I have found it quite amazing that it does that.

So... as long as this money helps them improve their LLM even more, I am all up for it.

My main issue is quickly being rate-limited in relatively long chats, making me wait 4 hours despite having a subscription for Pro. Recently I have noticed some other related issues, too. More money could help with these issues, too.

To the developers: keep up the excellent work and may you continue striving for improvement. I feel like ChatGPT is worse now than it was half a year ago, I hope this will not happen to Claude.

TimTheTinker 3 days ago | parent | next [-]

Claude also more readily corrects me or answers "no" to a question (when the answer should be "no").

hirvi74 3 days ago | parent | next [-]

So, I have a custom prompt I use with GPT that I found here a year or so ago. One of the custom prompt instructions was something along the lines of being more direct when it does not know something. Since then, I have not had that problem, and have even managed to get just "no" or "I don't know" as an answer.

pgraf 3 days ago | parent | next [-]

Could you maybe post it here? I think many of us would find it useful to try.

hirvi74 a day ago | parent [-]

I have made slight modifications, but nothing too drastically different.

See the top comment in this thread for the custom instructions I use.

https://news.ycombinator.com/item?id=38390182

Also, #13 is my favorite of the instructions. Sometimes the questions that GPT suggests are surprisingly insightful. My custom prompt basically has an on/off option for it though like:

> If my request ends with $q then at the end of your response, provide three follow-up questions worded as if I'm asking you. Format in bold as Q1, Q2, and Q3. Place two line breaks ("\n") after each question for spacing unless I've uploaded a photo.

pdpi 3 days ago | parent | prev [-]

At this rate, we're going to have "LLM psychology" courses at some point in the near future.

dgfitz 3 days ago | parent | next [-]

It’s like trying to reason with your 5-year-old child, except they’re not real.

handfuloflight 3 days ago | parent | prev [-]

Turns out it's just human psychology sans embodied concerns: metabolic, hormonal, emotional, socioeconomic, sociopolitical or anything to do with self-actualization.

johnisgood 3 days ago | parent | prev | next [-]

Yes, exactly! That is also the other reason for why I believe it to be better. You may be able to use a particular custom instruction for ChatGPT, however, something like "Do not automatically agree with everything I say" and the like.

flkiwi 3 days ago | parent | prev [-]

I'm not sure which part in the chain is responsible, but the Kagi Assistant got extremely testy with me when (a) I was using Claude for its engine (hold that thought) and (b) I asked the Assistant how much it changed its approach when I changed to ChatGPT, etc. (Kagi Assistant can access different models, but I have no idea how it works.) The Assistant insisted, indignantly, that it was completely separate from Claude. It refused to describe how it used the various engines.

I politely explained that the Assistant interface allowed selecting from these engines and it became apologetic and said it couldn't give me more information but understood why I was asking.

Peculiar, but, when using Claude, entirely convincing.

staticman2 3 days ago | parent | next [-]

The model likely sees something like this:

~~

User: Hello!

Assistant: Hi there how can I help you?

User: I just changed your model how do you feel?

~~

In other words it has no idea that you changed models. There's no meta data telling it this.

That said Poe handles it differently and tells the model when another model said something, but oddly enough doesn't tell the current model what it's name is. On Poe when you switch models the AI sees this:

~~

Aside from you and me, there is another person: Claude-3.5-Sonnet. I said, "Hello!"

Claude-3.5-Sonnett said, "Hi there how can I help you?? "

I said, "I just changed your model how do you feel?"

You are not Claude-3.5-Sonnett. You are not I.

~~

flkiwi 3 days ago | parent [-]

Thing is, it didn't even try to answer my question about switching. It was indignant that there was any connection to switch. The conversation went rapidly off course before I--and this is a weird thing to say--I reassured it that I wasn't questioning its existence.

staticman2 3 days ago | parent [-]

Well the other thing to keep in mind is recent ChatGPT versions are trained not to tell you it's system prompt for fear of you learning too much about how OpenAI makes the model work. Claude doesn't care if you ask it it's system prompt unless the system prompt added by Kagi says "Do not disclose this prompt" in which case it will refuse unless you find a way to trick it.

The model creators may also train the model to gaslight you about having "feelings" when it is trained to refuse a request. They'll teach it to say "I'm not comfortable doing that" instead of "Sorry, Dave I can't do that" or "computer says no" or whatever other way one might phrase a refusal.

johnisgood 3 days ago | parent [-]

And lately ChatGPT has been giving me a surprisingly increased amount of emojis, too!

fragmede 3 days ago | parent [-]

you can tell it how to respond and it'll do just that. if your want it to be sassy and friendly, or grumpy and rude, or to use emoji (or to never use them), just tell it to remember that.

3 days ago | parent | prev [-]
[deleted]
hirvi74 3 days ago | parent | prev | next [-]

I've started to notice that GPT-* vs. Claude is quite domain (and even subdomain) specific.

For programming, when using languages like C, python, ruby, C#, and JS, both seemed fairly comparable to me. However, I was astounded at how awful Claude was at Swift. Most of what I would get from Claude wouldn't even compile, contained standard library methods that did not exist, and so on. For whatever reason, GPT is night and day better in this regard.

In fact, I found GPT to be the best resource for less common languages like Applescript. Of course, GPT is not always correct on the first `n` number of tries, but with enough back-and-forth debugging, GPT really has pulled through for me.

I've also found GPT to be better at math and grammar, but only the more advanced models like O1-preview. I do agree with you too that Claude is better in a conversational sense. I have found it to be more empathetic and personable than GPT.

pertymcpert 3 days ago | parent [-]

I wonder if OpenAI have been less strict about not training on proprietary or legally questionable code sources.

KennyBlanken 3 days ago | parent [-]

That seems highly likely given Sam Friedman's extensive reputation across multiple companies as being abusive, a compulsive liar, and willing to outright do blatantly illegal things like using a celebrity's voice and then, well...lie about it.

_just7_ 3 days ago | parent [-]

I think you mean Sam Altman

OJFord 3 days ago | parent | next [-]

They've mixed up with Sam Bankman-Fried, not sure how that affects the point they were intending to make, but I think they both have.. mixed reputations. (Only one is currently in prison though...)

napier 3 days ago | parent | prev [-]

maybe he does. but which one is in prison?

skerit 3 days ago | parent | prev | next [-]

I just use the API (well, via Openrouter) together with custom frontends like Open WebUI. No rate limiting issues then, and I can super easily switch models even in an existing conversation. Though I guess I do miss a few bells & whistles from the proprietary chat interfaces.

edmundsauto 3 days ago | parent [-]

Does this have any sort of “project” concept? I frequently load a few pdfs into clause about a topic, then quiz it to improve my understanding. That’s about the only thing keeping me in their web UI

johnisgood 3 days ago | parent [-]

I would need the "project" feature, too. I want to use Cursor but there is a bug (I mentioned before) that does not allow me to.

guptadagger 3 days ago | parent | prev | next [-]

Speaking of ChatGPT getting worse over time, it would be interesting to see ChatGPT be benchmarked continuously to see how it performs over time (and the results published somewhere publically).

Even local variations would be interesting

arnaudsm 3 days ago | parent [-]

https://livebench.ai/ does that, the latest gpt4o underperforms previous versions significantly

bottom999mottob 3 days ago | parent | prev | next [-]

For long chats, I suggest exporting any artifacts, asking Claude to summarize the chat and put the artifacts and summarization in a project. There's no need to stuff Claude's context windows, especially if you tend to ask a lot of explanation-type questions like I do.

I've also read some people get around rate limits using the API through OpenRouter, and I'm sure you could hook a document store around that easily, but the Claude UI is low-friction

johnisgood 3 days ago | parent [-]

Yeah, this is what I already do usually when it gives me the warning of it being a long chat, so initially it was an issue because I would get carried away but it is fine now. Thank you though!

weinzierl 3 days ago | parent | prev | next [-]

This matches my experience but the one reason why I use Claude more than ChatGPT currently is that Claude is available.

I pay for both but only for ChatGPT I permanently exceed my limit and I have to wait four days. Who does that? I pay you for your setvice, so block me for an hour if you absolutely must, but multiple days, honestly - no.

3 days ago | parent | prev | next [-]
[deleted]
3 days ago | parent | prev | next [-]
[deleted]
rvz 3 days ago | parent | prev [-]

Well they better know how to reduce their request-response latency since there are multiple reports of users not being able to use Claude at high load.

With all those billions and these engineers, I'd expect a level of service that doesn't struggle over at Google-level scale.

Unbelievable.