| ▲ | convolvatron 35 minutes ago | |
this one may actually be solvable, but I kinda timed out on it. okay, so I'm doing distributed systems, and my primary abstraction is a large stream that I'm using to connect components across machines. its kind of mandatory that I implement back pressure for these streams, because awful things can happen if I don't. ok, I have a large local container of records that I want to stream over a back pressured channel. now we clearly have a problem that the iterator needs to be long lived, and I don't really want to serialize all accesses to the collection for a streaming operation that may never terminate. for this particular domain I don't actually care about serializability of the iterators view of the collection. I clearly don't want the container to be left in an erroneous or inconsistent state, but otherwise anything goes. all the iterators for all the standard rust containers have lifetimes bounded by the container (batch), and because they all represent internal state, they all have to be mutable. and in this case they need to be async also. I don't think there is an 'idiomatic' way to represent that access pattern, and it kind of necessitates writing ones own container. if there is a good answer to this I'd be curious to hear it, but I consider this one of the major personal failure modes for rust, which is 'oh, yeah, well, in order to figure that out to need to understand [long list of compiler instristic property types and runtime behaviour], which isn't super pragmatic. since I'm here, I've already wasted words here talking about the dismal async situation, but I think more important to me as a systems programmer is the lifetime abstraction. It think its great to put a name to it and try to put rules around lifetimes - these are traditionally implicit things that we reason about _outside the program text_, and that's a real bother. however everything I do is state management, and the lifetimes of those states (files, connections, higher level sessions) don't directly correspond to the lexical calling stack. the only tools that rust gives me is Arc, which some with a whole set of busy caveats, or trying to thread several lifetimes through the entire call path - a road that I've been down and abandoned for readability and maintenance. now you can argue that rust would prefer that I use a thread per object, and while that might be workable, that a pretty strong and side-effecting constraint to apply to all of the programs in my domain. anyways I ended up keeping the last key issued, and for every new record, paying the logn cost to get to approximately where I was perviously. which brings up another general complaint. as imperfect as it is, sometimes I want to just stand sometime up and look at it. I don't want to spend 3 weeks coming up with sharing policies and structures that work well with rust. I want to look at it and measure it and count its defects in my head and decide what to do next. in rust I kind of have to decide up front what I'm building, and while that might be an excellent thing to encourage in some contexts, I don't think it is in all contexts. | ||