| ▲ | pansa2 2 days ago |
| I wonder whether Raymond Hettinger has an opinion on this PEP. A long time ago, he wrote: "freezing dicts is a can of worms and not especially useful". https://mail.python.org/pipermail/python-dev/2006-February/0... |
|
| ▲ | pkulak 2 days ago | parent | next [-] |
| This is why I love how Rust approached this; almost by accident to make borrow checking work. Every reference is either mutable or not, and (with safe code), you can't use an immutable reference to get a mutable reference anywhere down the chain. So you can slowly construct a map through a mutable reference, but then return it out of a function as immutable, and that's the end of it. It's no longer ever mutable, and no key or value is either. There's no need to make a whole new object called FrozenHashMap, and then FrozenList, and FrozenSet, etc. You don't need a StringBuilder because String is mutable, unless you don't want it to be. It's all just part of the language. Kotlin _kinda_ does this as well, but if you have a reference to an immutable map in Kotlin, you are still free to mutate the values (and even keys!) as much as you like. |
| |
| ▲ | rcxdude 2 days ago | parent | next [-] | | Only if you're returning a reference or wrapping it in something that will only ever return a reference. If you return an object by value ('owned'), then you can do what you like with it and 'mut' is just an light guardrail on that particular name for it. | |
| ▲ | vlovich123 2 days ago | parent | prev | next [-] | | You cannot return an immutable version. You can return it owned (in which case you can assign/reassign it to a mut variable at any point) or you can take a mut reference and return an immutable reference - but whoever is the owner can almost always access it mutably. | | |
| ▲ | pkulak 2 days ago | parent | next [-] | | Arg, you’re right. Not sure what I was thinking there. I still think my point stands, because you get the benefits of immutability, but yeah, I didn’t explain it well. | |
| ▲ | tekne 2 days ago | parent | prev [-] | | I mean, if you return an immutable reference, the owner in fact cannot mutate it until that reference is dropped. If you in fact return e.g. an Rc::new(thing) or Arc::new(thing), that's forever (though of course you can unwrap the last reference!) | | |
| ▲ | aw1621107 a day ago | parent [-] | | > I mean, if you return an immutable reference, the owner in fact cannot mutate it until that reference is dropped. Might be worth noting that "dropped" in this context doesn't necessarily correspond to the reference going out of scope: fn get_first(v: &Vec<i32>) -> &i32 {
&v[0]
}
fn main() {
let mut v = vec![0, 1, 2];
let first = get_first(&v);
print!("{}", first});
v.push(3); // Works!
// print!("{}", first); // Doesn't work
}
|
|
| |
| ▲ | the__alchemist 2 days ago | parent | prev [-] | | My favorite part of rust is its explicit control over mutability, in the manner you describe. |
|
|
| ▲ | jonathaneunice 2 days ago | parent | prev | next [-] |
| That's a great link and recommended reading. It explains a lot about the design of Python container classes, and the boundaries of polymorphism / duck typing with them, and mutation between them. I don't always agree with the choices made in Python's container APIs...but I always want to understand them as well as possible. Also worth noting that understanding changes over time. Remember when GvR and the rest of the core developers argued adamantly against ordered dictionaries? Haha! Good times! Thank goodness their first wave of understanding wasn't their last. Concurrency and parallelism in Python was a TINY issue in 2006, but at the forefront of Python evolution these days. And immutability has come a long way as a design theme, even for languages that fully embrace stateful change. |
| |
| ▲ | zahlman 2 days ago | parent [-] | | > Also worth noting that understanding changes over time. Remember when GvR and the rest of the core developers argued adamantly against ordered dictionaries? Haha! Good times! The new implementation has saved space, but there are opportunities to save more space (specifically after deleting keys) that they've now denied themselves by offering the ordering guarantee. | | |
| ▲ | jonathaneunice 2 days ago | parent [-] | | Ordering, like stability in sorting, is an incredibly useful property. If it costs a little, then so be it. This is optimizing for the common case, where memory is generally plentiful and dicts grow more than they shrink. Python has so many memory inefficiencies that occasional tombstones in the dict internal structure is unlikely to be a major effect. If you're really concerned, do `d = dict(d)` after aggressive deletion. | | |
| ▲ | zahlman 2 days ago | parent | next [-] | | > Ordering, like stability in sorting, is an incredibly useful property. I can't say I've noticed any good reasons to rely on it. Didn't reach for `OrderedDict` often back in the day either. I've had more use for actual sorting than for preserving the insertion order. | | |
| ▲ | mcherm 2 days ago | parent | next [-] | | Personally, I find lots of reasons to prefer an orders Dict to an unordered one. Even small effects like "the debugging output will appear in a consistent order making it easier to compare" can be motivation enough in many use cases. | |
| ▲ | xen0 2 days ago | parent | prev | next [-] | | It's sometimes nice to be deterministic. I don't often care about a specific order, only that I get the same order every time. | | |
| ▲ | no_wizard 2 days ago | parent [-] | | Thinking about this upfront for me, I am actually wondering why this is useful outside of equality comparisons. Granted, I live and work in TypeScript, where I can't `===` two objects but I could see this deterministic behavior making it easier for a language to compare two objects, especially if equality comparison is dependent on a generated hash. The other is guaranteed iteration order, if you are reliant on the index-contents relationship of an iterable, but we're talking about Dicts which are keyed, but extending this idea to List, I see this usefulness in some scenarios. Beyond that, I'm not sure it matters, but I also realize I could simply not have enough imagination at the moment to think of other benefits | | |
| ▲ | xen0 2 days ago | parent | next [-] | | I work on a build system (Bazel), so perhaps I care more than most. But maybe it does all just come down to equality comparisons. Just not always within your own code. | |
| ▲ | dontlaugh 2 days ago | parent | prev [-] | | Being able to parse something into a dict and then serialise it back to the same thing is a bit easier. Not a huge advantage, though. |
|
| |
| ▲ | morshu9001 2 days ago | parent | prev | next [-] | | Same. Recently I saw interview feedback where someone complained that the candidate used OrderedDict instead of the built-in dict that is now ordered, but they'll let it slide... As if writing code that will silently do different things depending on minor Python version is a good idea. | | |
| ▲ | tehnub 2 days ago | parent | next [-] | | Well it's been guaranteed since 3.7 which came out in 2018, and 3.6 reached end-of-life in 2021, so it's been a while. I could see the advantage if you're writing code for the public (libraries, applications), but for example I know at my job my code is never going to be run with Python 3.6 or older. | | |
| ▲ | morshu9001 2 days ago | parent [-] | | Yeah, if you have that guarantee then I wouldn't fault anyone for using dict, but also wouldn't complain about OrderedDict. |
| |
| ▲ | sam_bristow a day ago | parent | prev [-] | | Honestly, if I was writing some code that depended on dicts being ordered I think I'd still use OrderedDict in modern Python. I gives the reader more information that I'm doing something slightly unusual. | | |
| |
| ▲ | vanviegen 2 days ago | parent | prev | next [-] | | Indeed! I don't understand why it isn't more common for stdlibs to include key-ordered maps and sets. Way more useful than insertion ordering. | | |
| ▲ | zahlman 2 days ago | parent [-] | | Presumably because it involves different performance characteristics. |
| |
| ▲ | kzrdude 2 days ago | parent | prev | next [-] | | It seems like opinions really differ on this item then. I love insertion sort ordering in mappings, and python with it was a big revelation. The main reason is that keys need some order, and insertion order -> iteration order is a lot better than pseudorandom order (hash based orders). For me, it creates more reproducible programs and scripts, even simple ones. | |
| ▲ | BiteCode_dev 2 days ago | parent | prev [-] | | Ordering is very useful for testing. This morning for example, I tested an object serialized through a JSON API. My test data seems to never match the next run. After a while, I realized one of the objects was using a set of objects, which in the API was turned into a JSON array, but the order of said array would change depending of the initial Python VM state. 3 days ago, I used itertools.group by to group a bunch of things. But itertools.group by only works on iterable that are sorted by the grouping key. Now granted, none of those recent example are related to dicts, but dict is not a special case. And it's iterated over regularly. |
| |
| ▲ | seanhunter 2 days ago | parent | prev | next [-] | | Ordering is specifically a property (useful or not) that a set doesn't have. You need a poset for it to be ordered. I would expect to use a different data structure if I needed an ordered set. | |
| ▲ | LtWorf 2 days ago | parent | prev [-] | | Does your code actually rely on that? I've never once needed it. |
|
|
|
|
| ▲ | mvanbaak 2 days ago | parent | prev | next [-] |
| This was 19 (almost) 20 years ago.
As stated in the lwn.net article, a lot of concurrency has been added to python, and it might now be time for something like a frozendict. Things that were not useful in 2006 might be totally useful in 2026 ;P Still, like you, I'm curious wether he has anything to say about it. |
| |
| ▲ | aewens 2 days ago | parent [-] | | I think Raymond Hettinger is called out specially here because he did a well known talk called [Modern Dictionaries](https://youtu.be/p33CVV29OG8) where around 32:00 to 35:00 in he makes the quip about how younger developers think they need new data structures to handle new problems, but eventually just end up recreating / rediscovering solutions from the 1960s. “What has been is what will be, and what has been done is what will be done; there is nothing new under the sun.” | | |
| ▲ | kzrdude an hour ago | parent | next [-] | | I think he was always reluctant to add features, and his version of Python would be slimmer, beautiful, and maybe 'finished'. His voice is definitely not guiding the contemporary Python development, which is more expansionist in terms of features. | |
| ▲ | sesm 2 days ago | parent | prev [-] | | Since that time HAMT was invented and successfully used in Scala and Clojure, so this talk didn't age well. | | |
|
|
|
| ▲ | dkarl 2 days ago | parent | prev | next [-] |
| It's interesting that he concludes that freezing dicts is "not especially useful" after addressing only a single motivation: the use of a dictionary as a key. He doesn't address the reason that most of us in 2025 immediately think of, which is that it's easier to reason about code if you know that certain values can't change after they're created. What a change in culture over the last 20 years! |
| |
| ▲ | morshu9001 2 days ago | parent [-] | | You can't really tell though. Maybe the dict is frozen but the values inside aren't. C++ tried to handle this with constness, but that has its own caveats that make some people argue against using it. | | |
| ▲ | krick 2 days ago | parent [-] | | Indeed. So I don't really understand what this proposal tries to achieve. It even explicitly says that dict → frozendict will be O(n) shallow-copy, and the contention is only about O(n) part. So… yeah, I'm sure they are useful for some cases, but as Raymond has said — it doesn't seem to be especially useful, and I don't understand what people ITT are getting excited about. | | |
| ▲ | 2 days ago | parent | next [-] | | [deleted] | |
| ▲ | morshu9001 2 days ago | parent | prev [-] | | Maybe treating Python like a systems language, so applying the same reasoning for const in C++ and Rust to it |
|
|
|
|
| ▲ | zahlman 2 days ago | parent | prev | next [-] |
| > Another PEP 351 world view is that tuples can serve as frozenlists; however,
that view represents a Liskov violation (tuples don't support the same
methods). This idea resurfaces and has be shot down again every few months. ... Well, yes; it doesn't support the methods for mutation. Thinking of ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance relationship. I normally find Hettinger very insightful so this one is disappointing. But nobody's perfect, and we change over time (and so do the underlying conditions). I've felt like frozendict was missing for a long time, though. And really I think the language would have been better with a more formal concept of immutability (e.g. linking it more explicitly to hashability; having explicit recognition of "cache" attributes, ...), even if it didn't go the immutable-by-default route. |
| |
| ▲ | kccqzy 2 days ago | parent | next [-] | | Apple (or perhaps NeXT) has solved this problem already in Objective-C. Look at NSArray and NSMutableArray, or NSData and NSMutableData. It’s intuitive and Liskov-correct to make the mutable version a subclass of the immutable version. And it’s clearly wrong to have the subclass relationship the other way around. Given how dynamic Python is, such a subclass relationship need not be evident at the C level. You can totally make one class whose implementation is independent of another class a subclass of the other, using PEP 3119. This gives implementations complete flexibility in how to implement the class while retaining the ontological subclass relationship. | |
| ▲ | pansa2 2 days ago | parent | prev | next [-] | | > ImmutableFoo as a subclass of Foo is never going to work. And, indeed, `set` and `frozenset` don't have an inheritance relationship. Theoretically, could `set` be a subclass of `frozenset` (and `dict` of `frozendict`)? Do other languages take that approach? > linking [immutability] more explicitly to hashability AFAIK immutability and hashability are equivalent for the language's "core" types. Would it be possible to enforce that equivalence for user-defined types, given that mutability and the implementation of `__hash__` are entirely controlled by the programmer? | | |
| ▲ | kccqzy 2 days ago | parent | next [-] | | Yes you could. Other languages do. See NSMutableSet and NSSet in Objective-C. | |
| ▲ | chriswarbo 2 days ago | parent | prev [-] | | > Theoretically, could `set` be a subclass of `frozenset` (and `dict` of `frozendict`)? At one extreme: sure, anything can be made a subclass of anything else, if we wanted to. At the other extreme: no, since Liskov substitution is an impossibly-high bar to reach; especially in a language that's as dynamic/loose as Python. For example, consider an expression like '"pop" in dir(mySet)' | | |
| ▲ | tremon 2 days ago | parent [-] | | > consider an expression like '"pop" in dir(mySet)' class frozenset:
pass
class set(frozenset):
def pop(self, key):
pass
I don't see why hasattr(mySet, 'pop') should be a problem here? | | |
| ▲ | chriswarbo 2 days ago | parent [-] | | > I don't see why hasattr(mySet, 'pop') should be a problem here? I never said it's a problem (and I never said it's not!). I was specifically addressing two things: - The "theoretical" nature of the question I quoted (i.e. ignoring other aspects like subjectivity, practicality, convention, etc.) - The reasoning about "Liskov violation", which was quoted further up this thread. For context, here's Liskov's definition of their principle (from https://en.wikipedia.org/wiki/Liskov_substitution_principle ): > Barbara Liskov and Jeannette Wing described the principle succinctly in a 1994 paper as follows:[1] > > Subtype Requirement: Let ϕ(x) be a property provable about objects x of type T. Then ϕ(y) should be true for objects y of type S where S is a subtype of T. My expression `"pop" in dir(mySet)` gives an explicit example of how `set` and `frozenset` are not subtypes of each other (regardless of how they're encoded in the language, with "subclasses" or whatever). In this case `ϕ(x)` would be a property like `'"pop" in dir(x)' = 'False'`, which holds for objects x of type frozenset. Yet it does not hold for objects y of type set. Your example of `hasattr(mySet, 'pop')` gives another property that would be violated. My point is that avoiding "Liskov violations" is ("theoretically") impossible, especially in Python (which allows programs to introspect/reflect on values, using facilities like 'dir', 'hasattr', etc.). (FYI I became rather jaded on the Liskov substitution principle after reading https://okmij.org/ftp/Computation/Subtyping ) | | |
| ▲ | kccqzy 2 days ago | parent | next [-] | | > I became rather jaded on the Liskov substitution principle after reading https://okmij.org/ftp/Computation/Subtyping The root of the issue here is that Liskov substitution principle simply references ϕ(x) to be some property satisfied by objects of a class. It does not distinguish between properties that are designed by the author of the class to be satisfied or properties that happen to be satisfied in this particular implementation. But the Hyrum’s Law also states that properties that are accidentally true can become relied upon and as time passes become an intrinsic property. This to me suggests that the crux of the problem is that people don’t communicate sufficiently about invariants and non-invariants of their code. | |
| ▲ | tremon 2 days ago | parent | prev [-] | | > > Subtype Requirement: Let ϕ(x) be a property provable about objects x of type T. Then ϕ(y) should be true for objects y of type S where S is a subtype of T. This says "if hasattr(parent, 'pop') == True then hasattr(child, 'pop') must be True". This is not violated in this case, since hasattr(parent, 'pop') is False. If you want to extend the above definition so that negative proofs concerning the parent should also hold true for the child, then subtyping becomes impossible since all parent and child types must be identical, by definition. | | |
| ▲ | minitech 2 days ago | parent [-] | | The property in question is `hasattr(x, "pop") is False`. > If you want to extend the above definition so that negative proofs concerning the parent should also hold true for the child, then subtyping becomes impossible since all parent and child types must be identical, by definition. The distinction isn’t “negative proofs”, but yes, that’s their point. In Python, you have to draw a line as to which observable properties are eligible. |
|
|
|
|
| |
| ▲ | FreakLegion a day ago | parent | prev | next [-] | | > I've felt like frozendict was missing for a long time, though. Type the dict as a mapping when you want immutability: x: Mapping[int, int] = {1: 1}
x[1] = 2 # Unsupported target for indexed assignment ("Mapping[int, int]").
The only problem I've seen with this is: y = {}
y[x] = 0 # Mypy thinks this is fine. Mapping is hashable, after all!
The issue here is less that dict isn't hashable than that Mapping is, though. | | |
| ▲ | zahlman a day ago | parent [-] | | This is because the ABC system is defined such that MutableMapping is a subtype of Mapping. Which mostly makes sense, except that if we suppose there exist Mappings that aren't MutableMappings (such that it makes sense to recognize two separate concepts in the first place), then Mapping should be hashable, because immutable things generally should be hashable. Conceptually, making something mutable adds a bunch of mutation methods, but it also ought to take away hashing. So Liskov frowns regardless. | | |
| ▲ | FreakLegion 18 hours ago | parent [-] | | It really doesn't make sense for there to be an inheritance relationship between Mapping and MutableMapping if Mapping is immutable (it isn't, of course), but the weirder part is still just that the typing machinery is cool with unhashable key types like: x: dict[list, int] = {}
x[[1, 2, 3]] = 0
|
|
| |
| ▲ | tmp10423288442 2 days ago | parent | prev | next [-] | | And, to the point of this proposal, `dict` and `frozendict` don't have an inheritance relationship either. | |
| ▲ | immibis 2 days ago | parent | prev [-] | | ImmutableFoo can't be a subclass of Foo, since it loses the mutator methods. But nor can Foo be a subclass of ImmutableFoo, since it loses the axiom of immutability (e.g. thread-safety) that ImmutableFoo has. When you interpret Liskov substitution properly, it's very rare that anything Liskov-substitutes anything, making the entire property meaningless. So just do things based on what works best in the real world and aim for as much Liskov-substitution as is reasonable. Python is duck-typed anyway. It's a decent guiding principle - Set and ImmutableSet are more substitutable than Set and Map, so Set deriving from ImmutableSet makes more sense than Set deriving from Map. It's just not something you can ever actually achieve. |
|
|
| ▲ | ndr 2 days ago | parent | prev | next [-] |
| Immutability it's a joy to work with. Ask anyone who's worked with Clojure's dicts. |
|
| ▲ | morshu9001 2 days ago | parent | prev [-] |
| I agree, same with frozenset. If you really want to use one of those as a key, convert to a tuple. There might be niche use cases for all this, but it's not something that the language or even the standard lib need to support. |
| |
| ▲ | boothby 2 days ago | parent [-] | | Problem being that sets aren't consistently ordered and conversion to a tuple can result in an exponential (specifically, factorial) explosion in the number of possible keys associated with a single set. Nor can you sort all objects. Safe conversion of sets to tuples for use as keys is possible but the only technique I know requires an auxiliary store of objects (mapping objects to the order in which they were first observed), which doesn't parallelize well. | | |
| ▲ | morshu9001 2 days ago | parent [-] | | tuple(sorted(s)) and if you can't even sort the values, they're probably not hashable. I get that this involves a copy, but so does frozenset, and you can cross that bridge in various ways if it's ever a problem. | | |
| ▲ | boothby 2 days ago | parent [-] | | Here are some types that support hashing: str
bytes
int, float
complex
tuple
frozenset
Aside from int and float, you cannot perform comparisons between objects of different types. Moreover, you cannot sort complex numbers at all.I have crossed that bridge, and I'm telling you (again) that a sorted tuple is not a generic solution. | | |
| ▲ | morshu9001 2 days ago | parent [-] | | I'm not saying the problem with tuple doesn't exist, but that there doesn't need to be a built-in way to deal with it. If for some unfortunate reason you've got a mixed-type set that you also want to use as a dict key, you can write a helper. |
|
|
|
|