Biff.graph: structure your Clojure codebase as a queryable graph

It's very cool you managed to make a mini Pathom - esp in so few lines of code :))

But the end result looks almost identical? Resolver declarations are a bit reorganized and look a bit cleaner - though you could do that with a wrapper around Pathom. Why not fork Pathom and just make some QOL adjustments?

If you are okay only having a subset, then I think you could streamline Pathom quite a bit more.. Off the top of my head:

- adding and "registering" resolvers in to an environment is always annoying. To me this feels like it should be abstracted away and you should never has to manage this stuff. Each time you add a resolver you have to copy the new resolver name, scroll down to the bottom of the file and paste it in to the resolver list. Stale resolvers floating around in the ns and forgetting to register resolvers regularly leads to weird scenarios where you're wondering why something isn't resolving. I think the user shouldn't really have to think about any of this.. the environment should be constructed automatically by the engine. Scan the namespace and register ever resolver.

I get that it'd be not as flexible this way.. but I've never had to make multiple environments in one namespace.

- You should be able to safely reregister resolvers. This happens when you try to decouple sub-systems that depend on common resolvers. Ex: I have some utility resolvers that do some format conversions. If I add them to the environments of two namespaces, then those two namespaces can't be registered in a parent namespace. It's not a dealbreaker, you just register all your ns environments all the top-level and do all your queries there. But you can't add inline test queries in these lower level namespaces and it sort of breaks the decoupling (esp if it's across library boundaries).

- The Pathom errors are actually pretty good once you know how to read them (but there is a ton of visual noise). My guess is it's going to be a challenge to get to the same level in a rewrite. It feels like there is room for improvement here, but I don't have concrete ideas. Maybe a ASCII diagram of the chain of missing keys? There is Pathom-Viz, but from what I understand it doesn't handle nested queries (which in a not-toy program will be basically all your queries)

▲

chamomeal 2 hours ago | parent | prev | next [-]

> biff.graph is basically a lightweight version of Pathom. It implements only a subset of Pathom's functionality with the intention of being easier to understand.

I’ve heard of pathom but I’ve never actually dove in and tried it out. It sounds super neato though.

I have a bunch of microservice DB’s (that should really just be on DB, but I think we’re created in the peak of microservices hype). I def need a better way to explore the data. “Easier to understand” sounds sick and I think I’ll check it out this week!

▲

adamfeldman 9 hours ago | parent | prev | next [-]

There are a few great talks on YouTube by the creator of Pathom explaining the value of the graph: https://m.youtube.com/watch?v=IS3i3DTUnAI

▲

geokon 2 hours ago | parent [-]

I've been rewriting a codebase using Pathom resolvers and it has been extremely fun and has made me really reexamine how I organize code. Without being hyperbolic, it's really a new coding paradigm. You get some extreme decoupling and it allows the engine to automatically maximizes concurrency.

▲

jacobobryant 2 hours ago | parent [-]

Yeah, it's a big eye-opener. I'd like to see if I can figure out an ergonomic way to do it in Python since I do a fair amount of work in that, and passing ORM objects around isn't great.

	▲	brianritz 2 hours ago \| parent [-]
		I have an old repo that explores the concept at bmritz/datajet. I’ve also toyed around with the idea of using type hints and type aliases in python as the “data key” (equivalent to :user/id). Would love to have something equivalent in python.

▲

Capricorn2481 2 hours ago | parent | prev | next [-]

My experience with Pathom, and other graph query libraries, is it feels like a deliberately confusing way to reason about a program. I'd like to know your thoughts on it.

From what I hear, the main draw is separating what you want from how you get it, so your calling code can just focus on what it needs. But you can use regular functions to do that. What libraries like Pathom do is leave it open to the caller what shape of data they need.

But I think letting the caller do subtle query changes that can completely change which resolvers are triggered and how something is fetched is kinda leaky. How do you write the perfect resolver for all situations? How do you keep them from accidentally exploding their fetches? Is it not better to have things be explicit through function calls instead of chasing down disjointed call graphs?

	▲	geokon an hour ago \| parent [-]
		You can use regular functions, but there are several things you lose: - intermediate keys are not recalculated if they're used across different resolvers. This means you basically never need to manage caches of precomputed results. So if you're calling `my-func` on `input-a` everywhere, you don't need to do all the ceremony of computing it once, storing it somewhere, and then passing it around to everyone that needs it. It's all just handled automatically. Code simplifies greatly - It's much easier to "inject" lower-level steps b/c resolvers are essentially declaring an interface. If you suddenly don't like your interface and want a new interface, then you make a new interface and a bridging resolver. Refactoring is much easier. If you want to introduce an entirely new input format that usually just involves adding a single new resolver that outputs the inputs to your system (at whatever part of the pipeline you want). While with a pipeline of function calls it's generally more messy. It hard to make a generalization here b/c it depends on how your functions are organized. - With the async engine you can automatically resolve branches concurrently without having to manage or think about it. You get a lot less stalls in the code. I haven't really hit an "exploding their fetches" scenario personally. Things like optional inputs and resolvers that rely on precedence rules are generally a bit of a code smell and are usually points where I start to think about how to reorganize my code

▲

tosti 7 hours ago | parent | prev [-]

The name suggests it plots graphs related to incoming email.