| ▲ | 20k 9 hours ago |
| How is this faster than just reading the documentation? Given that LLMs hallucinate, you have to double check everything it says against the docs anyway |
|
| ▲ | subscribed 9 hours ago | parent | next [-] |
| I learn fastest from the examples, from application of the skill/knowledge - with explanations. AIs allowed me to get on with Python MUCH faster than I was doing myself, and understand more of the arcane secrets of jq in 6 months than I was able in few years before. And AIs mistakes are brilliant opportunity to debug, to analyse, and to go back to it saying "I beg you pardon, wth is this" :) pointing at the elementary mistakes you now see because you understand the flow better. Recently I had a fantastic back and forth with Claude and one of my precious tools written in python - I was trying to understand the specifics of the particular function's behaviour, discussing typing, arguing about trade-offs and portability. The thing I really like in it that I always get a pushback or things to consider if I come up with something stupid. It's a tailored team exercise and I'm enjoying it. |
|
| ▲ | simonw 9 hours ago | parent | prev | next [-] |
| Human teachers make mistakes too. If you aren't consuming information with a skeptical eye you're not learning as effectively as you could be no matter what the source is. The trick to learning with LLMs is to treat them as one of multiple sources of information, and work with those sources to build your own robust mental of how things work. If you exclusively rely on official documentation you'll miss out on things that the documentation doesn't cover. |
| |
| ▲ | 20k 9 hours ago | parent [-] | | If I have to treat LLMs as a fallible source of information, why wouldn't I just go right to the source though? Having an extra step in between me and the actual truth seems pointless WinAPI docs are pretty accurate and up to date | | |
| ▲ | simonw 9 hours ago | parent | next [-] | | Because it's faster. If the WinAPI docs are solid you can do things like copy and paste pages of them into Claude and ask a question, rather then manually scan through them looking for the answer yourself. Apple's developer documentation is mostly awful - try finding out how to use the sips or sandbox-exec CLI tools for example. LLMs have unlocked those for me. | | |
| ▲ | 20k 7 hours ago | parent [-] | | But you have to check the answer against the documentation anyway though, to validate that its actually correct! Unless you're just taking the LLM answers at face value? | | |
| ▲ | simonw 7 hours ago | parent [-] | | For most code stuff you don't check the answer against the documentation - you write the code and run it and see if it works. That's always a better signal than anything that official documentation might tell you. | | |
| ▲ | 20k 6 hours ago | parent [-] | | That seems like a strong error, you have no idea if it works or if it just happens to work | | |
| ▲ | simonw 5 hours ago | parent [-] | | If you're good at programming you can usually tell exactly why it worked or didn't work. That's how we've all worked before coding agents came along too - you don't blindly assume the snippet you pasted off StackOverflow will work, you try it and poke at it and use it to build a firm mental model of whether it's the right thing or not. | | |
| ▲ | 20k 5 hours ago | parent [-] | | Sure. A big part of how I'd know that the function I'm calling does what I think it does, is by reading the source documentation associated with it Does it have any threading preconditions? Any weird quirks? Any strange UB? That's stuff you can't find out just by testing. You can ask the LLM, but then you have to read the docs anyway to check its answer | | |
| ▲ | simonw 3 hours ago | parent [-] | | I envy you for the universally high quality of documentation that the code you are working with has! |
|
|
|
|
|
| |
| ▲ | mgraczyk 9 hours ago | parent | prev [-] | | Because it will take you years to read all the information you can get funneled through an LLM in a day | | |
| ▲ | 20k 7 hours ago | parent [-] | | Except you have no idea if what the LLM is telling you is true I do a lot of astrophysics. Universally LLMs are wrong about nearly every astrophysics questions I've asked them - even the basic ones, in every model I've ever tested. Its terrifying that people take these at face value For research at a PhD level, they have absolutely no idea what's going on. They just make up plausible sounding rubbish | | |
| ▲ | cdetrio 6 hours ago | parent | next [-] | | Astrophysicist David Kipping had a podcast episode a month ago reporting that LLMs are working shockingly well for him, as well as for the faculty at the IAS.[1] It's curious how different people come to very different conclusions about the usefulness of LLMs. https://youtu.be/PctlBxRh0p4 | | |
| ▲ | 20k 6 hours ago | parent [-] | | The problem with these long videos is that what I really want to see is what questions were asked of it, and the accuracy of the results Every time I ask LLMs questions I know the answers to, its results are incomplete, inaccurate, or just flat out wrong much of the time The idea that AI is an order of magnitude superior to coders is flat out wrong as well. I don't know who he's talking to |
| |
| ▲ | mgraczyk 7 hours ago | parent | prev [-] | | Somehow we went from writing software apps and reading API docs to research level astrophysics Sure it's not there yet. Give it a few months | | |
| ▲ | 20k 6 hours ago | parent [-] | | It doesn't even work for basic astrophysics I asked chatgpt the other day: "Where did elements heavier than iron come from?" The answer it gave was totally wrong. Its not a hard question. I asked it this question again today, and some of it was right (!). This is such a low bar for basic questions |
|
|
|
|
|
|
| ▲ | mgraczyk 9 hours ago | parent | prev [-] |
| Yes you have to be careful, but the LLM will read and process core and documentation literally millions of times faster than you, so it's worth it |
| |
| ▲ | 20k 7 hours ago | parent | next [-] | | I mean, is it really that hard to find information in the docs? Like, if I want to find out what, I don't know, "GetQueuedCompletionStatus" does. I google GetQueuedCompletionStatus Find this page: https://learn.microsoft.com/en-us/windows/win32/api/ioapiset... Bam, that's the single source of truth right there. Microsoft's docs are pretty great If I use an LLM, I have to ask it for the documentation about "GetQueuedCompletionStatus". Then I have to double check its output, because LLMs hallaucinate Doubly checking its output involves googling "GetQueuedCompletionStatus", finding this page: https://learn.microsoft.com/en-us/windows/win32/api/ioapiset... And then reading the docs to validate whether or not what its told me is correct. How does this save me any time? | | |
| ▲ | mgraczyk 7 hours ago | parent [-] | | How about we do the following. I have not done win32 programming in 12 years. Maybe you've done it more recently. I'll use an LLM and you look up things manually. We can see, who can build a win32 admin UI that shows a realtime view of every open file by process with sorting, filtering and search on both the files and process/command names. I estimate this will take me 5 minutes
Would you like to race? | | |
| ▲ | 20k 6 hours ago | parent [-] | | This mentality is fundamentally why I think AI is not that useful, it completely underscores everything that's wrong with software engineering and what makes a very poor quality senior developer I'll write an application without AI that has to be maintained for 5 years with an ever evolving featureset, and you can write your own with AI, and see which codebase is easiest to maintain, the most productive to add new features to, and has the fewest bugs and best performance | | |
| ▲ | mgraczyk 6 hours ago | parent [-] | | Sure let's do it. I am pretty confident mine will be more maintainable, because I am an extremely good software engineer, AI is a powerful tool, and I use AI very effectively I would literally claim that with AI I can work faster and produce higher quality output than any other software engineer who is not using AI. Soon that will be true for all software engineers using AI. | | |
| ▲ | 20k 5 hours ago | parent [-] | | I'm curious, have you ever worked on a single software project for more than 5 years? |
|
|
|
| |
| ▲ | skydhash 7 hours ago | parent | prev [-] | | Why does it matter? We have table of contents, index and references for books and other contents. That’s a lot of navigational aid. Also they help in providing you a general overview of the domain. |
|
