The DeepWiki tool itself seems pretty neat. It has a pretty good go at collecting documentation from across the codebase and organising it in one place. It has a pretty good guess at coming up with documentation which isn't there.

It strikes me as an example of automated code assistance that's e.g. more useful than "the item under the cursor has type <X>, here's its documentation".

There are things which benefit from being automatically described, and there are things where "the map is not the territory", and you do want someone to have come up with a map.

> "Treat it like a patient senior engineer."

I trust that LLMs are patient (you can ask them stupid questions without consequence).

I do not trust LLMs to act as 'senior'. (i.e. Unless you ask it to, it won't push back against dumb ideas, or suggest better ideas that would achieve what you're trying to do. -- And if you just ask it to 'push back', it's going to push back more than necessary).

▲

ignoramous 7 days ago | parent | next [-]

> It strikes me as an example of automated code assistance that's e.g. more useful than "the item under the cursor has type <X>, here's its documentation". ... I trust that LLMs are patient (you can ask them stupid questions without consequence).

DeepWiki does add tremendous value already: I maintain open source projects and frequently direct volunteers to use DeepWiki to explore those (fairly convoluted) codebases. But ... I've caught DeepWiki hallucinating pretty convincingly far more than once just because a struct / a package / a function was named for something it wasn't doing anymore / wasn't doing it by the book (think: RFCs, docs, specifications etc). This isn't a criticism more than it is of the refactoring practices by the maintainers themselves. "Code readability" and tests (part of "gyms" for Agents), then I imagine, are going to come in clutch for OSS projects that want to benefit from a constant stream of productive external contributions.

	▲	jcranmer 7 days ago \| parent [-]
		I'm pretty dubious that the value it adds is in fact positive. I first thought to look at DeepWiki for a large project--LLVM, specifically--and the results ranged from incomplete to just plain incorrect (see https://news.ycombinator.com/item?id=45020628 for more details). Okay, to be fair, maybe looking at a multimillion line codebase is just too much for an AI tool, although it's arguably one of the use cases they're gunning for. For a smaller project, I rummaged through compiler-explorer since I once poked around in that codebase. And when looking through its description of the property files (specifically https://deepwiki.com/compiler-explorer/compiler-explorer/3.3...), I noticed that it has some very subtly incorrect description of what they do, the kind of mistake that's likely to boomerang on you only a month or so later. I then decided to see the quality of the ask-a-question system. At this point, I happened to be poking around CLP from COIN-OR trying to gauge how accurate it was about the simplex details, and I noticed it mentioned pivot tolerance here: https://deepwiki.com/coin-or/Clp/2.4-factorization-and-linea... . Playing a newbie, and given that it doesn't really explain pivot tolerance, I asked it to explain it in detail. Now the correct answer is essentially as follows: > When you are factoring a matrix A, the algorithm is essentially dividing a column by a particular value in that column (the "pivot"), and then adding that column to every other column in the matrix. To minimize the accumulation of error in the calculation (i.e., for "numerical stability"), you want that value to be as large (in magnitude, i.e., ignoring sign) as possible. When you are working with sparse matrices, you have another problem, though: you want to avoid converting a zero entry into a nonzero entry ("fill-in"), and the pivots that minimize the amount of nonzero entries you create aren't necessarily the ones that minimize numerical error, and thus there's a tension between these two goals. The pivot tolerance factor is the ratio of the smallest-magnitude value you are willing to accept to the largest-magnitude value, which controls how much numerical stability you are willing to sacrifice for the goal of minimizing fill-in. [And there's a natural follow-up question, what value is reasonable here, for which I have yet to find a satisfying answer.] This was DeepWiki's attempt at answering the question and some follow-ups: https://deepwiki.com/search/what-does-pivot-tolerance-mean_7... . It doesn't really attempt to explain what it does beyond a handwave towards "numerical stability" (there's no discussion of sparsity, for example), instead wanting to focus on how it's adaptive. It also has the general feeling of a student trying to pad their paper to meet a minimum length requirement. Finally, it doesn't answer the question "what is the default value" (I think the answer is 0.1, though I don't know the codebase well enough to confirm that the places where it's setting that are the actual initial values). At pretty much every step of the process, DeepWiki has given me answers that are distinctly worse than what I would have found just traipsing through the code myself, and given the need to confirm whatever an AI says for correctness given their propensity for hallucination, it's hard for me to say that on those occasions where it was correct that it would have been faster. I do realize that navigation of large code bases is a skill that seems to be in short supply, but I have yet to see evidence that AI is good at covering for that.

▲

giancarlostoro 6 days ago | parent | prev [-]

I'm trying it out with a repo that has no code comments or documentation. Let's see how it does. :)

Edit: its been like over 10 minutes, and still nothing, thats interesting, I did choose a Lingo source project so it probably gave up by now.