Running Shell Commands in Python (the right way)

Sometimes your Python script needs to break out of Python's ecosystem. You need Docker to build an image, git to check repository status, or ffmpeg to process media files. When Python libraries aren't enough, you're forced to shell out.

The question becomes: how do you run shell commands cleanly without making your Python code fragile?

The Problem: Shell Commands That Don't Play Nice

You're building a deployment script that needs to verify your Git repository is clean before proceeding:

import os

# Quick and dirty
os.system("git status --porcelain")

This runs, but you can't capture the output to actually check if the repo is clean. You can't handle errors gracefully. If the command fails, your script just continues anyway.

You need shell commands that integrate with your Python logic, not work against it.

The Solution: Choose the Right subprocess Tool

Python's subprocess module gives you two main approaches, each suited for different scenarios.

For scenarios where you need to run the command, get the result, and move on, use subprocess.run():

import subprocess

result = subprocess.run(
    ['git', 'status', '--porcelain'], 
    capture_output=True, 
    text=True, 
    check=True
)

if result.stdout.strip():
    print("Repository has uncommitted changes")
    exit(1)

The advantage: You get output capture, error handling, and security (no shell injection) in a straightforward API.

However, for a more complex workflow (such as streaming output or fine-grained control), use subprocess.Popen():

import subprocess

process = subprocess.Popen(
    ['docker', 'build', '.'], 
    stdout=subprocess.PIPE, 
    stderr=subprocess.STDOUT,
    text=True
)

for line in process.stdout:
    print(f"BUILD: {line.strip()}")

if process.wait() != 0:
    raise subprocess.CalledProcessError(process.returncode, 'docker build')

The advantage: You see output as it happens and can interact with long-running processes.

When to use subprocess.run() vs subprocess.Popen()

Use subprocess.run() when:

• You need the command to finish before continuing

• You want simple output capture and error handling

• The command completes quickly (under a few seconds)

Use subprocess.Popen() when:

• You need real-time output streaming

• You're running long processes (builds, transfers, compilations)

• You need to pipe commands together or terminate processes early

Avoid os.system() unless you're prototyping and genuinely don't care about output or error handling.

The choice between subprocess.run()and subprocess.Popen() comes down to whether you need the results immediately or want to stream the process. Make the right call and your shell integration becomes clean and predictable.

📧 Join the Python Snacks Newsletter! 🐍

Want even more Python-related content that’s useful? Here’s 3 reasons why you should subscribe the Python Snacks newsletter:

1. Get Ahead in Python with bite-sized Python tips and tricks delivered straight to your inbox, like the one above.

2. Exclusive Subscriber Perks: Receive a curated selection of up to 6 high-impact Python resources, tips, and exclusive insights with each email.

3. Get Smarter with Python in under 5 minutes. Your next Python breakthrough could just an email away.

You can unsubscribe at any time.

Interested in starting a newsletter or a blog?

Do you have a wealth of knowledge and insights to share with the world? Starting your own newsletter or blog is an excellent way to establish yourself as an authority in your field, connect with a like-minded community, and open up new opportunities.

If TikTok, Twitter, Facebook, or other social media platforms were to get banned, you’d lose all your followers. This is why you should start a newsletter: you own your audience.

» Use this link to get 20% off for 3 months your newsletter! «

This article may contain affiliate links. Affiliate links come at no cost to you and support the costs of this blog. Should you purchase a product/service from an affiliate link, it will come at no additional cost to you.