Odd, I don't see any mention of subprocess.run, the workhorse of python scripting.

Quick rundown for the unfamiliar:

Give it a command as a list of strings (e.g., subprocess.run(["echo", "foo"]).)

It takes a bunch of flags, but the most useful (but not immediately obvious) ones are:

  check=True: Raise an error if the command fails
  capture_output=True: Captures stdout/stderr on the CompletedProcess
  text=True: Automatically convert the stdout/stderr bytes to strings

By default, subprocess.run will print the stdout/stderr to the script's output (like bash, basically), so I only bother with capture_output if I need information in the output for a later step.

▲ bccdee a day ago | parent | next [-]

Also `asyncio.subprocess`, which lets you manage multiple concurrently running commands. Very handy if you need to orchestrate several commands together.

▲ theomega 10 hours ago | parent | prev | next [-]

One thing I can recommend that makes scripting in python with external commands a lot easier is the `sh` module:

https://pypi.org/project/sh/

Basically you can just `from sh import [command]` and then have an installed binary command available as function

  from sh import ifconfig
  print(ifconfig("eth0"))

▲ theamk 5 hours ago | parent | next [-]

Please don't use "sh" python library!

By default (1) captures stdout and stderr of all processes and (2) create tty for processs stdout.

Those are really bad defaults. The tty on stdout means many programs run in "interactive" rather then "batch" mode: programs which use pager get output truncated, auto-colors may get enabled and emit ESC controls into output streams (or not, depending on user's distro... fun!). And captured stderr means warnings and progress messages just disappear.

For example, this hangs forever without any output, at least if executed from interactive terminal:

    from sh import man
    print(man("tty"))

Compare to "subprocess" which does the right thing and returns manpage as a string:

    import subprocess
    subprocess.check_output(["man", "tty"], text=True)

Can you fix "sh"? sure, you need to bake in option to disable tty. But you've got to do it in _every_ script, or you'll see failure sooner or later. So it's much easier, not to mention safer, to simply use "subprocess". And as a bonus, one less dependency!

(Fun fact: back when "sh" first appeared, everyone was using "git log" as an example of why tty was bad (it was silently truncating data). They fixed it.. by disabling tty only for "git" command. So my example uses "man" :) )

▲ Lvl999Noob 5 hours ago | parent | prev | next [-]

And for the opposite, where you keep your main pipeline in shell but want to use python for some parts of it, there is pypyp.

https://pypi.org/project/pypyp/

It takes cares of the input and output boilerplate so you can focus on the actual code that you wanted python for.

    > seq 1 5 | pyp 'sum(map(int, lines))'
    > ls | pyp 'Path(x).suffix'

▲ grumps 8 hours ago | parent | prev | next [-]

`sh` is nice but it requires a dependency. No dependencies is nicer IMHO. uv makes this way easier but for low dependency systems, or unknown environments stdlib is king.

▲ btown 6 hours ago | parent | prev | next [-]

I love how this import trick shows how hackable Python is - and it’s this very hackability that has led to so many of the advances we see in AI. Arguably without operator overloads we’d be 5 or more years behind.

https://github.com/amoffat/sh/blob/2a90b1f87a877e5e09da32fd4...

▲ pxc 9 hours ago | parent | prev | next [-]

I've used Plumbum for this for some projects at work, and really like it for this.

https://plumbum.readthedocs.io/en/latest/local_commands.html...

It also does argument parsing and validation, so it's generally pretty useful for writing little CLI tools that invoke other CLI tools.

https://plumbum.readthedocs.io/en/latest/cli.html

▲ zelphirkalt 5 hours ago | parent | prev | next [-]

Does this give live output (meaning before completion) of processes run?

▲ code_biologist 10 hours ago | parent | prev [-]

uv for using sh as a dependency in scripts, managed inline, has changed it from “eh, I’ll just use subprocess” to “why not” for me.

https://docs.astral.sh/uv/guides/scripts/#using-different-py...

	▲	grumps 8 hours ago \| parent [-]
		I love uv, the why not works great if it’s your machine but in places without your machine uv is just another step for a customer.

▲ albertzeyer 10 hours ago | parent | prev [-]

I think the point is that for most things, you don't need to call any external tools. Python's standard library comes already with lots of features, and there are many packages you can install.