Remix.run Logo
Show HN: I estimated the carbon impact of different LLMs(modelpilot.co)
8 points by aposded a day ago | 4 comments

(did my best with the data that is available online)

niek_pas a day ago | parent | next [-]

This is neat. It's a little hard to compare the best-performing models ("~ 0.00006232g" vs. "~0.00013746g" is not easy at a glance) --- perhaps better to use scientific notation, milligrams, or CO2e per 10M tokens?

aposded 21 hours ago | parent [-]

You're right! Fixed.

jraph a day ago | parent | prev [-]

Did you include the environmental impact of training these models? I failed to find the information and that seems quite important to me (I'm a bit tired so maybe that's just me).

> Our proprietary methodology and algorithms are trade secrets and confidential business information

Sorry, but that's a no go IMHO. This stuff needs to be reproducible, transparent, ideally peer reviewed, and we must be able to study it. You are doing the equivalent of "here are facts, just trust us". That's not reliable.

What is it you are selling that such a core thing like your methodology is kept secret?

I tried to understand a bit what you did reading your methodology page [1] but didn't understand a thing. I looked for concrete stuff but didn't find any. Now, again, I'm a bit tired so I'm possibly missing something.

We need to see for ourselves all this stuff to determine if you are rigorous, if there are possible conflict interests (for what I know, you could be selling good grades), etc. The basic requirements of scientific research, basically.

[1] https://modelpilot.co/methodology

myaccountonhn 21 hours ago | parent [-]

I agree with the rest of your comment, but w.r.t. training impact, it's not necessarily such a big factor in emissions[1]

[1] https://limited.systems/articles/climate-cost-of-ai-revoluti...