Very interesting article and good points.

But how about we flip the original statement around:

Many problems thought to require giant software packages/libraries are very solvable locally if you know what you are doing

This is why LC is actually meaningful - imagine if you faced the coin challenge problem IRLin prod and you decided to pull in a constraint solver - what would've been a 25 line function now is a giant python library that carries 30MB of native dependencies around.

These solutions are often lacking in many ways, since mentally offloading some of the work means you don't have to understand the problem in depth or how the designated tool solves it, which can lead to nasty surprised in production.

We see this everywhere - people pulling in questionable npm packages to save themselves 30 mins of thinking or 20 mins of reading docs - people convinced that you do need that huge 3D framework that comes with an editor to make a WebGL widget on your landing page etc.

You are more willing to accept bloat of others, just because they pulled in Electron, because they were too afraid to learn how native UI works.

People need to be more curious, and strive to be more knowledgeable.

▲

jakewins 4 days ago | parent | next [-]

You are right to be sceptical of “dependency bloat” - many many systems could be made simpler to debug and maintain if that advice was followed.

But: I urge you to give constraint solvers a try. This is something very different from some hipster ORM library. The good ones will be a single dependency containing extremely carefully optimised matrix math code that let you solve optimisation problems faster and with much more maintainable code.

Try some toy problem with the HiGHS solver for instance, it has good python bindings and even ships as a wasm package you can run in browsers. “Find the optimal times to run this heat pump given electricity prices”, “find the ideal train departure times that optimise for when people need to travel and minimises transfers”

▲

jvanderbot 4 days ago | parent | prev | next [-]

Wait are you making the opposite claim? That one should eschew the "correct" formulation in favor of a bespoke one? Despite the stated (and hopefully obvious) difficulties that brings with maintenance, generalization, etc?

▲

mmis1000 4 days ago | parent | next [-]

You probably haven't see front-end projects that pulls tons of library for a simple sorting or grouping task. Sometimes even solvable with build-in array function alone. It's a true nightmare when you have to deal with that kind of projects.

▲

komali2 4 days ago | parent [-]

What, like lodash? Some of those hail from a time that we didn't have a good set of native methods. So the library is just legacy'd in. But I do agree, lodash performance compared to native functions is crazy bad.

▲

mwcz 4 days ago | parent [-]

Is that the case? I saw a talk by the author of lodash years ago and he touched on performance. The built-in functions are (or were?) implemented in JS, level terrain for a library like lodash to beat "native" performance. Lodash beat browser built-ins in some cases. The talk was ten years ago, though, so things may have changed. Perhaps more of the built-ins are written in C++ now.

Here's the talk: https://youtu.be/2DzaOnOyCqE?si=McCMjzGopzSCoaCi

▲

mmis1000 3 days ago | parent [-]

Ｉt's not really about the performance. It's about the readability. The performance don't even matter if you are just grouping dozens or hundreds items on the gui.

Abuse Lodash or Underscore everywhere means the code is not readable unless you know what is every used lodash function intended to do. Combing with weak type and undocumented business logic (and api types), it's a true nightmare experience to work with.

It's probably irrelevant in a personal project. But it's a headache if you are collaborating.

Sometimes it got even worse that original user don't know the exact specification of the utility function either. And the code works just by pure luck. (something like, the method work if all keys are numeric) It will give you a hard time to troubleshoot the problem if it happens.

▲

mwcz 3 days ago | parent [-]

"It's not really about the performance" the parent comment was about the performance of lodash.

▲

mmis1000 2 days ago | parent [-]

That's exact why I reply this though (or probably I should reply to your parent comment?). The place these get used are mostly performance insensitive location, it don't really do anything except ruins the readability. And the project is not that old either(It's a past es6-era project), I don't really get why it is used at all.

	▲	mwcz 2 days ago \| parent [-]
		I'm not a lodash stalwart or anything, but years ago when I used it heavily, it made navigating and manipulating large/complex/irregular objects much easier. It's LINQ for JavaScript. And it was very readable and consistent, since most functions in lodash have the same signature.

▲

torginus 4 days ago | parent | prev [-]

I'm sorry but after reading your comment I can't seem to be able to decide if you favor writing the dynamic programming version or pulling in the constraint solver.

Both are correct in the sense that they give the right output, and I don't think pulling in a huge library (maintained by who knows and for how long) is going to be beneficial for maintenace. And having a good understanding of both the precise requirements, and the algorithms used to solve them also helps maintenance.

It's just having two dozen lines tucked of code away in a function in the repo seems infinitely more maintainable to me then using some giant framework (of possibly unknown quality) to solve the issue.

This is a general argument I'm making, not just applying to this constraint solver/

▲

bubblyworld 3 days ago | parent [-]

There are other aspects to maintenance, like requirements change. In this case it's trivial to change or add new constraints to a constraint solver, whereas even small changes to a typical DP problem can require a total rethink of the approach. Extending the analogy to other kinds of dependencies left as an exercise for the reader.

Point being that software has many dimensions. Reducing the use of dependencies to fear of learning or thinking is a bit reductive in my opinion, even for stuff that seems simple initially.

▲

torginus 3 days ago | parent [-]

Imo, deepending on the desired quality of the result and the amount and complexity of bespoke requirements, the more of the former are present, the more strongly I consider rolling something bespoke.

With out-of-the-box libraries, the more custom requirements I have, the more trouble I tend to have supporting them, and trying to make something do a thing it wasn't designed for, can erase initial gains very quickly. At least this has been my experience over the years.

	▲	bubblyworld 3 days ago \| parent [-]
		I mean, it really depends. I'm writing models for some energy traders at the moment, and things were simple enough in the mvp that we could hand-roll our optimisations. But when we started having to map real contracts it got out of hand super fast. Linear constraints solvers have been a godsend, it makes adding new types of contacts and constraints really easy (and supports almost arbitrary kinds of optimisations). It's just never as easy as "don't do this" or "always do this". But yeah, I agree with your gist, right tools for the right job and all that. Most of the time you don't know all requirements up front and it can make sense to hedge your bets in the interest of speed until you do.

▲

hirvi74 4 days ago | parent | prev | next [-]

I agree with essentially everything you said. While reinventing the wheel isn't always to most efficient solution, such actions have sometimes paid dividends in my past.

> People need to be more curious, and strive to be more knowledgeable.

Absolutely, though I am not sure lack of curiosity nor strive is always the issue. It's one thing to green field one's own solution in an attempt to better one's abilities, but it's also a bit idealistic in a working world of impending deadlines and death-march Agile sprints.

In this case, a 25 line function to solve a constraint would likely take me far less time than grokking an external library's documentation, but in many cases, I feel developers jump on external dependencies because time is of the essence.

▲

DoctorOetker 3 days ago | parent | prev [-]

hard disagree:

Given ANY problem: first ask yourself have others solved this before? that is a hard question to answer, since we don't know in which context a similar puzzle was solved before (structures in different domains can boil down to the same mathematical puzzle). A literature search would be very time costly.

The most important point is the flexibility in being able to change the puzzle (a small change in the puzzle can result in a large change in the type of solution), as the author of the article points out. The bespoke algorithm is brittle. Description of the problem itself is less brittle (you can reuse most of the problem statement).

It may sound incredibly expensive to pull in a constraint solver, but if the application warrants constraint-solver-quality results, it should afford either the dependency, or the data structures and solvers used in the solver dependency, to optimize for the application,

Its just bizarre to ask people to beat the feats of those standing on the shoulders of giants, without allowing them to stand on the shoulders of giants too.

Think about why one is recruiting employees with data structure skills (so more than just information plumbers). Is it really so strange that the most qualified people understand the reality that the state of the art is constantly changing, but understand at least the basics of how these solvers work internally?

Viewed through this lens, the ideal job candidate are those who design, implement and maintain... constraint solvers! Assuming their familiarity with the constraint-solver code-base they could profile the solver package while its solving the puzzle. Do this on many instances of the puzzle, and keep track of the dead-ends and optimal solutions, to figure out which functionality can be ripped out of the solver, and which must be kept.

So in order of preference:

1. Programmers or mathematicians (or equivalent, think physicists, etc.) familiar with 1 or more constraint solver source code bases.

2. End-users of a constraint solver package, with sufficient familiarity (as a user) with constraint solvers. WITH data structure and algorithms experience.

3. End-users of a constraint solver package as above, but without data structure and algorithms knowledge

4. People with data structure and algorithms knowledge.

If 1. is too expensive you'll have to combine skills over multiple hires.

If industry is really interested in good profiles for these criteria, it should sponsor universities / students to get familiar with constraint solver usage, and if possible constraint solver development and contributions. After a few years the candidates will pop into existence.

	▲	Avshalom 3 days ago \| parent [-]
		>>It may sound incredibly expensive to pull in a constraint solver Like this is actually kinda the point of the article. On every constraint programming article half the comments insist that they could do the (example) problem in a dozen lines of C so bringing in SCIP or OR-Tools is "too much" for the (example) problem. Wayne's point (here and in other articles he's written) is that actually constraint problems are fucking everywhere. This is one of the real lessons of learning Prolog: data queries, scheduling, SAT, LP, integer programming, optimization... etc; it's all constraint programming, hell parsing is a constraint problem if you do it right. No one sees it like that though so they see their entire business application logic as containing at most one knapsack.