Remix.run Logo
Python type hints may not be not for me in practice(utcc.utoronto.ca)
42 points by ingve 14 hours ago | 55 comments
solidsnack9000 a few seconds ago | parent | next [-]

It seems like the author is looking for the ability to specify types as `typeof <function>:arguments` and `typeof <function>:return`. I can see how this could make prototyping easier. It is also helpful for cases (not uncommon in Python) where you're just proxying another function.

aSanchezStern 31 minutes ago | parent | prev | next [-]

The thing that the author says they would prefer is already in Python, it's called NewType (https://docs.python.org/3/library/typing.html#typing.NewType)

They say "...so I can't create a bunch of different names for eg typing.Any and then expect type checkers to complain if I mix them."

`MyType = NewType('MyType', Any)`

is how you do this.

At the end, they suggest a workflow: "I think my ideal type hint situation would be if I could create distinct but otherwise unconstrained types for things like function arguments and function returns, have mypy or other typing tools complain when I mixed them, and then later go back to fill in the concrete implementation details of each type hint"

That's just doing the above, but then changing the `NewType('MyType', Any)` to something like `NewType('MyType', list[dict[str, int]])` later when you want to fill in the concrete implementation.

tasn 8 hours ago | parent | prev | next [-]

The Python type system is pretty bad, but it's still 100x better than not using types. We are heavy users of the (Rust) type system at Svix, and it's been a godsend. I wrote about it here https://www.svix.com/blog/strong-typing-hill-to-die-on/

We also use Python in some places, including the shitty Python type-system (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).

belorn 5 hours ago | parent | next [-]

Looking at that blog post, I find it illustrative in how people who like strong types and people who dislike strong types are addressing different form of bugs. If the main types of issues comes from bugs like 1 + "2" == 12", then strong types is a big help. It also enables many developers who spend the majority of time in a programming editor to quickly get automatic help with such bugs.

The other side is those people who do not find those kind of bugs annoying, or they simply don't get hit by such bugs at a rate that is high enough to warrant using a strong type system. Developers who spend their time prototyping in ipython also get less out of the strong types. The bugs that those developers are concerned about are design bugs, like finding out why a bunch of small async programs reading from a message buss may stall once every second Friday, and where the bug may be a dependency of a dependency of a dependency that do not use a socket timeout. Types are similar not going to help those who spend the wast majority of time on bugs where someone finally says "This design could never have worked".

teddyh 28 minutes ago | parent | next [-]

Take care to differentiate strong/weak typing from dynamic/static typing. Many dynamically typed languages (especially older ones) are also weakly typed, but some dynamic langugages, like Python, are strongly typed. 1 + "2" == 12 is weak typing, and Python has strong typing. Type declarations are static typing, in contrast to traditional Python, which had (and still has) dynamic typing.

satvikpendem 30 minutes ago | parent | prev | next [-]

It's not about the bugs, it's about designing the layout of the program in types first (ie, laying out all of the data structures required) such that the actual coding of the functionality is fairly trivial. This is known as type driven development: https://blog.ploeh.dk/2015/08/10/type-driven-development/

tasn 2 hours ago | parent | prev [-]

I think you're missing the point of the blog a bit, as `1 + "2" == "12"` style of issues wasn't it. It definitely also sucks and much more common than you make it sound (especially when refactoring) but it's definitely not that.

Anyhow, no need to rehash the same arguments, there was a long thread here on HN about the post, you can read some of it here: https://news.ycombinator.com/item?id=37764326

ansgri 5 hours ago | parent | prev | next [-]

If it’s 100x better than no types, then probably 10x better than C++ type system. It takes some time to unlearn using dicts everywhere, but then namedtuples become your best friend and noticeably improve maintainability. Probably the only place where python type system feels inadequate is describing json-like data near the point of its (de)serialization.

youdontknowjuli 8 hours ago | parent | prev | next [-]

> (and some cool hackery to make SQLAlchemy feel very typed and work nicely with Pydantic).

Sounds interesting. Can you elaborate on the cool hackery? We introduced SQLModel recently but struggle in a few cases (e.g. multi-level joins). Do you know reference projects for SQLAlchemy and pydantic?

tasn 2 hours ago | parent [-]

My info is maybe a bit dated, as it's been a while since we wrote this hackery. We also adopted SQLModel at some point but we had to patch it to work well (I think some of my contributions are now in upstream). As for some of the hacks:

    def c(prop: t.Any) -> sa.Column:  # type: ignore
        return prop
To make it possible to access sqlmodel properties as columns for doing things like `in_` but still maintaining type safety.

Added types ourselves to the base model like this:

    __table__: t.ClassVar[sa.Table]
Added functions that help with typing like this:

    @classmethod
    async def _fetch_one(cls: t.Type[BaseT], db: BaseReadOnlySqlSession, query: Select) -> t.Optional[BaseT]:
        try:
            return (await db.execute(query)).scalar_one()
        except NoResultFound:
            return None
and stuff like this for relationships:

    def ezrelationship(
        model: t.Type[T_],
        id_our: t.Union[str, sa.Column],  # type: ignore
        id_other: t.Optional[t.Union[t.Any, sa.Column]] = None,  # type: ignore
    ) -> T_:
        if id_other is None:
            id_other = model.id
        return sqlm.Relationship(sa_relationship=relationship(model, primaryjoin=f"foreign({id_our}) == {id_other}"))


    def ezrelationship_back(
        id_our: t.Union[str, sa.Column],  # type: ignore
        id_other: t.Union[str, sa.Column],  # type: ignore
    ) -> t.Any:
        model, only_id2 = id_other.split(".")
        return sqlm.Relationship(
            sa_relationship=relationship(
                model,
                primaryjoin=f"foreign({id_our}) == {id_other}_id",
                back_populates=only_id2,
            )
        )

I hope this helps, I don't have time to find all the stuff, but we also hacked on SQLAlchemy a bit, and in other places.
Myrmornis 7 hours ago | parent | prev [-]

Can you give some examples of how the Python type system is disappointing you?

CJefferson 9 minutes ago | parent | next [-]

Mainly, the seems to be no way, in a dynamic language, to dynamically check if functions get the right types.

To me, this means I don't really understand the python type hinting at all, as adding hints to just one or two functions provides no value to me at all.

I assume I must be not using them usefully, as I've tried adding type hints to some projects and they just seemed to do nothing useful.

Spivak 3 minutes ago | parent [-]

Type hints alone don't do this, but you can use Pydantic to accomplish what you want. In Python type hints aren't enforced anywhere at runtime. They're for a type-checker to validate your source.

https://docs.pydantic.dev/latest/concepts/validation_decorat...

colemannerd 29 minutes ago | parent | prev [-]

default values! Since type hints are *hints*, it is difficult to set default values for complicated types. For instance, if you have lists, dicts, sets in the type signature, without a library like pydantic, it is difficult and non-standard. This becomes even more problematic when you start doing more complicated data structures. The configuration in this library starts to show the problems. https://koxudaxi.github.io/datamodel-code-generator/custom_t...

The issue very much is a lack of a standard for the entire language; rather than it not being possible.

alfons_foobar 12 minutes ago | parent [-]

I might be dense, but I don't understand what that has to do with type hints...

To my eyes, the problem of choosing useful defaults for complicated types/datastructures is independent of whether I add type hints for them.

I think I am missing something...

craftkiller 5 hours ago | parent | prev | next [-]

> In writing this it occurs to me that I do often know that I have distinct types (for example, for what functions return) and I shouldn't mix them, but I don't want to specify their concrete shape as dicts, tuples, or whatever. [...] Type aliases are explicitly equivalent to their underlying thing, so I can't create a bunch of different names for eg typing.Any and then expect type checkers to complain if I mix them.

It sounds to me like you're describing the NewType pattern which is just slightly farther down the page you linked in the article.

https://docs.python.org/3/library/typing.html#newtype

agubelu 13 hours ago | parent | prev | next [-]

Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.

I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.

Even worse, the IDE marks as an error some code that is perfectly correct, because it somehow doesn't match those nonsensical hints. And so it gives you the worst of both worlds: a lot of false positives that you quickly learn to ignore, dooming the few actual type errors to irrelevance, because you'll ignore them anyways until they blow up at runtime, just as it'd happen without typehints.

Hackbraten 11 hours ago | parent | next [-]

> Type hints are nice, until you have to interact with a library that isn't type-hinted, and then it very quickly becomes a mess.

Whenever I find myself in that situation, I usually write a typing stub for the parts that I use from that library (example: [0]) and then let `mypy_path` point to that directory [1].

VS Code will then pick up the hints from those stubs.

[0]: https://github.com/claui/itchcraft/blob/5ca04e070f56bf794c38...

[1]: https://github.com/claui/itchcraft/blob/5ca04e070f56bf794c38...

coldtea 8 hours ago | parent | prev | next [-]

>I don't know how other IDEs behave, but VScode + the Python extensions try to infer the missing hints and you end up with beauties such as `str | None | Any | Unknown`, which of course are completely meaningless.

Are they correct? If they're correct (even though they are a superset of the actual intended type) then what's the problem?

At worst, it's like not having type checks for that particular package.

wenc 5 hours ago | parent [-]

They are verbose but correct. I've caught some errors this way.

I usually don't think of None as a potential return value (= voids in C) but the LSP code analysis usually picks up on code paths that don't return a value.

I don't find Python's typing valuable for Jupyter type explorations, but they're immensely valuable for catching little issues in production code.

mcdeltat 31 minutes ago | parent | prev | next [-]

I believe there's a mode for VS Code type checking which ignores untyped code - have you tried that?

zelphirkalt 12 hours ago | parent | prev | next [-]

For example in mypy the default is to not check procedures, which have no argument type annotations and no return type annotation. That gets rid of your whole problem of untyped library, if you have a wrapper procedure.

If VSCode still highlights it, then it is time to configure VSCode properly.

thebigspacefuck 7 hours ago | parent | prev | next [-]

I believe VSCode by default uses pyright which is fast but shitty in that it gives a lot of false positives. If you want the most correct typing experience, use mypy. Even then you may need a config.

Lutger 12 hours ago | parent | prev | next [-]

I get what you need, yet I find these cases aren't all that often, and when it happens it doesn't bother me as I quickly recognize where the type system is somewhat failing and either ignore it or add a type hint.

But maybe if you have a codebase with a lot of magic of certain libraries, you experience is different. I also don't really depend on the typing treat it the same as C# or Java.

robjwells 12 hours ago | parent | prev [-]

Worst of both worlds is right. I came back to a Python project with a couple of critical but untyped dependencies recently after writing mostly Rust, and to clear up a large number of these (particularly “type is partially unknown”) I had the choice between lots of purely type-checking ceremony (`typing.cast`) or going without.

gizmo385 7 hours ago | parent | next [-]

The third option here is writing type stubs for the library, which you can sometimes find community versions of as well. They’re not too time consuming to write and generally work well enough to bridge the gap

robjwells 5 hours ago | parent [-]

Yeah, I think this may be a good option when actively working on a project. Sadly at the moment, it's mostly a case of "I just need to make a couple of bug fixes in this old project, why is my editor shouting at me?"

rrr_oh_man 11 hours ago | parent | prev [-]

What did you end up choosing & why?

robjwells 10 hours ago | parent [-]

It's only a personal side project and I have a good handle on the untyped modules in question, so in the end I suppressed most of the errors with `# type:ignore` and friends.

I'd reconsider that if I was doing more than the odd bug fix on the project. I still like Python, and started using type hints early, but there's enough added friction to make me question using them in the future.

I imagine on big projects the benefit is clearer.

rrr_oh_man 6 hours ago | parent [-]

Thanks for sharing!

Asking because I was really, really annoyed by the non-helpfulness of the type hints in practice, contrary to the theory.

greatgib 13 hours ago | parent | prev | next [-]

The logic of type hint is not bad but sadly I think that type hint are making python source code messy and unreadable.

I'm missing a lot simple functions with explicit argument names and docstrings with arguments types and descriptions clearly but discreetly documented.

It was one big strength of Python to have so simple and clean code without too much boilerplate.

Also, I have the feeling that static typing extremist are trying to push the idea that type hinting allows to ensure to not mix types as it would be bad. But from my point of view the polymorphic and typing mixing aspect is a strong force of Python.

Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.

One part where I find type hint to be wonderful still is for things like pydantic and dataclasses!

Vampiero 12 hours ago | parent | next [-]

> Like having dictionaries that are able to hold whatever you want is so incredible when you compare to trying to do the equivalent in Java for example.

Can't you just make a dictionary of objects, same as in C#? Except that in C#, if you really want to, you can also use `dynamic` to get python-like behavior.

Otherwise, generally speaking, in a strongly typed language you want to figure out what those objects have in common and put that inside an interface. If you can't modify those objects just slap an adapter pattern on top.

The result is a dictionary of objects that adhere to a specific interface, which defines all the properties and procedures that are relevant to the domain and the problem.

This makes thinking about the problem much easier from a type theoretical perspective because it lets you abstract away the concrete details of each object while preserving the fundamental aspects that you care about.

I guess that it takes two different mindsets to in order to appreciate the pros and cons of dynamic vs static programming. There are certainly many pros for dynamic programming, but I'm more comfortable thinking about problems in generic terms where every relation and constraint is laid bare in front of me, one level removed from the actual implementation.

wenc 5 hours ago | parent | prev | next [-]

> The logic of type hint is not bad but sadly I think that type hint are making python source code messy and unreadable.

Compared to legacy Python, yes.

Compared to verbose language like Java, no. Python typing is equal or less verbose than Java (unless you use "var" in Java).

mcdeltat 28 minutes ago | parent [-]

Python people legitimately upset they can't write every function like this now:

def func(data, *kwargs): """data: the data. kwargs: other data."""

Myrmornis 7 hours ago | parent | prev [-]

Python has union types, and you can type something as a container type with no type parameters.

greatgib 4 hours ago | parent [-]

You can but it defeats the purpose of typing. Makes a little bit more complicated to code and more verbose for almost no benefit. That is my point.

scoofy 4 hours ago | parent | prev | next [-]

My objection to strong types in python is philosophical. Python mimics natural language, and natural language rejects strong types for context resolved ambiguity.

In the way we resolve these issues in natural language, we can resolve bugs in python, that is, “do you mean integer ’3’ or string’3’” instead of insisting we define everything always forever.

To me, people who use type hinting are just letting me know they have written code that doesn’t check in line.

stoperaticless 2 minutes ago | parent | next [-]

Strong/weak types and static/dynamic typing are orthogonal things.

Strong type system limits things like 1+”1”.

Static type system requires type declarations (“int i”).

Python always had strong dynamic types.

orf an hour ago | parent | prev [-]

Python has always been strongly typed, since the very beginning.

The article and the feature has nothing to do with strong types.

truculent 8 hours ago | parent | prev | next [-]

The problem (in my opinion) is that Python gives you the tools (and perhaps even encourages you) to write code that would benefit from typing.

It's perfectly feasible to write maintainable, well-designed code in a dynamic language. I've worked with some extremely robust and ergonomic Clojure codebases before, for example. However, in Clojure, the language pushes you into its own "pit of success".

Personally, I never feel that with Python.

KaiserPro 12 hours ago | parent | prev | next [-]

Type hinting in python is a bit of a sticky plaster.

We have pyre enforcement at work, the problem is, that it has been gradually turned on over time, so some stuff is pyre compliant (or just strategically ignored) and some stuff isnt, so when you open some old code to do something, you have a million errors to deal with.

That would be fine if types were enforceable. in runtime type hinting does shit all.

I would dearly love a "strict" mode where duck typing is turned off and variable are statically typed. However I suspect that will never happen, even though it'd speed up a load of stuff if done correctly (type inference happens a lot)

I suspect to use type hints properly, I'd need to think a bit more C-like and create dataclasses as types to make things more readable, rather than using Dict[str,int] or what ever.

zelphirkalt 12 hours ago | parent [-]

There are some other ways of expressing names for types, once you start using typing. There are typevars, enums and using the "|" to separate options, there are TypedDict, NamedTuple, Union, Literal, Optional, and probably more. Not everything needs to be a dataclass.

thebigspacefuck 7 hours ago | parent | prev | next [-]

This is what we ended up using with mypy so we could add type checks to our CI without having to fix every single typing error:

https://github.com/orsinium-labs/mypy-baseline

travisgriggs 8 hours ago | parent | prev | next [-]

I typehint the stuff that is easy. My observations about typehinting in Python track the 80:20 rule or even a 90:10 rule. You get about 80% benefit for typhinting the easy 20%.

Hackbraten 13 hours ago | parent | prev | next [-]

> yet another Python thing I'd have to try to keep in my mind despite it being months since I used them last.

Typing hints are also a moving target: they have changed, sometimes significantly, on every minor Python release since they came to be.

The `Optional` type came and went (being replaced by the new Union syntax.) Type classes are usually born in the `typing` module, then some of them get moved to `collections`, `abc`, or `collections.abc`. Some types have even been moved several times. `TypeAlias` came and went (replaced by `type`). `List` became `list`. `Tuple` became `tuple`. Forward declarations require a workaround by using string literals. But in Python 3.14, they no longer do, and the workaround will become deprecated.

I'm an evangelist for static type checking, and I never write a Python function signature without complete typing annotations. I also think that typing annotations in Python are evolving in a good way. However, for long-lived Python scripts, I highly recommend that developers are aware of, and factor in, the effort necessary to keep up with the evolution of static typing across Python versions. That effort spent on migrating is going to come on top of the mental load that typing hints already add to your plate.

sesuximo 8 hours ago | parent | prev | next [-]

> PPS: I think my ideal type hint situation would be if I could create distinct but otherwise unconstrained types for things like function arguments and function returns, have mypy or other typing tools complain when I mixed them

Cleaner/safer function args and return types is a common motivation for dataclass. has benefits over many/complex args besides typing too.

dzonga 8 hours ago | parent | prev | next [-]

this is not a just a python problem but a problem in many dynamic languages.

switching to the typed variants whether typecript, python type-hints, mypy etc will force you to do the dance to make the compiler happy instead of working on code.

which is why for me - JSDoc is really good - it uses types for documentation and ends there.

terminalbraid 8 hours ago | parent | prev | next [-]

Sometimes I feel like we need an analog to javascript/typescript. Ptypethon if you will.

ansgri 5 hours ago | parent [-]

Absolutely. The main problem with python typing is that checking types is optional. A dialect with mandatory types (with inference) and runtime/load-time checking would be great.

solarkraft 8 hours ago | parent | prev | next [-]

It’s kind of cool that type hints can be reflected on to do things (see Pydantic). Other than that I find it pretty cumbersome to use in practice, coming from TypeScript. Semi-relatedly I also dislike Python’s different ways to access objects/dicts, it feels arbitrary and cumbersome.

andrewstuart 13 hours ago | parent | prev | next [-]

Program the way you like it and it should be fun.

If you are at work doing professional programming then typing helps avoid bugs and makes programming more of a reliable and robust process.

But doing your own thing, doing little utilities, banging out quick stuff, do exactly what makes you happy.

Programming should be fun not a chore and if types make it a chore for you then drop them.

aegis4244 13 hours ago | parent | prev | next [-]

Is that a double negative in your title ? Or is it an inside joke I didn't get ?

thebigspacefuck 7 hours ago | parent | next [-]

Ragebait title to get you to click it

DemocracyFTW2 13 hours ago | parent | prev [-]

You can't never have not enough of no good thing!

BiteCode_dev 13 hours ago | parent | prev [-]

That's why they are not unspecified to be optional.