Mastering Python Script Execution in Linux: From Command Line to Automation

Python’s role on Linux isn’t limited to ad-hoc utilities. Consider: data processing pipelines, alerting agents, automated admin tasks—nearly every serious engineering environment uses Python scripts somewhere.

Below, you’ll find a practical field guide to running, optimizing, and scheduling Python scripts on modern Linux systems. Focus is on Ubuntu 22.04+ and Python 3.9–3.11, but techniques largely apply across distributions.

Immediate Script Execution

If you have hello.py:

print("Hello, Linux user.")

Run it directly:

python3 hello.py

On some distributions, python still points to Python 2.x. Explicitly use python3 (verify with python3 --version). On RHEL/CentOS, you may need to install Python 3 via dnf install python3.

Shebang Method

Add a shebang for direct execution:

#!/usr/bin/env python3
print("Direct execution enabled.")

chmod +x hello.py
./hello.py

Known issue: If your default shell environment is non-standard, the shebang can misbehave (especially with old /usr/bin/python symlinks).

Multiple Python Versions

Mixed environments are common—legacy apps, system scripts, modern user projects. For example:

which python
which python3
python --version
python3 --version

To force a specific interpreter (v3.10):

/usr/bin/python3.10 script.py

Side note: Containers (Docker, Podman) are increasingly used to sidestep on-host version conflicts.

Dependency Isolation with venv

Global pip installs can break system tooling. Use project-specific environments:

python3 -m venv venv-test
source venv-test/bin/activate
pip install requests flask==2.2.5

Inside the venv, which python now points to .../venv-test/bin/python.

Deactivate when done:

deactivate

Trade-off: System-wide security updates don’t trickle into venvs. Manually rebuild environments after major OS upgrades.

Argument Handling

Scripts run non-interactively. Accept parameters, fail fast if incompatible. Example:

#!/usr/bin/env python3
import argparse

parser = argparse.ArgumentParser(description="User greeting script")
parser.add_argument('name', help='Name to greet')
args = parser.parse_args()

print(f"Hello, {args.name}")

Invocation:

./greet.py sysadmin

For complex CLIs: prefer argparse over ad-hoc sys.argv checking—better error messages, simpler testing.

Scheduling Scripts

Cron

Set up a script for routine execution (e.g., backup at 01:30):

crontab -e

Append:

30 1 * * * /usr/bin/python3 /opt/scripts/backup.py >> /var/log/backup.log 2>&1

Gotcha: Cron runs with a limited environment—PATH and virtualenvs may not work as expected. Always specify absolute paths.

systemd Timers

Systemd timers are now preferred on most modern distros due to dependency management, logging, and better lifecycle integration.

Example unit in /etc/systemd/system/disk-report.service:

[Unit]
Description=Generates daily disk usage report

[Service]
Type=oneshot
ExecStart=/usr/bin/python3 /opt/scripts/disk_usage.py

Timer in /etc/systemd/system/disk-report.timer:

[Unit]
Description=Runs disk-report.service daily

[Timer]
OnCalendar=*-*-* 04:00:00
Persistent=true

[Install]
WantedBy=timers.target

Commands:

systemctl daemon-reload
systemctl enable --now disk-report.timer

Check logs:

journalctl -u disk-report.service --since "1 hour ago"

Note: Timers persist missed runs across reboot with Persistent=true. Not all distros enable user timers without additional setup.

Logging & Observability

Headless execution means silent failure is a risk.

Integrate Python’s logging:

import logging

logging.basicConfig(
    filename='/var/log/my_script.log',
    level=logging.INFO,
    format='%(asctime)s %(levelname)s %(message)s'
)
logging.info("Script execution started")

Cron output can be mailed to MAILTO=user@example.com in crontab. Systemd uses journalctl—don’t expect print() output to show up in standard user logs.

Example log entry produced:

2024-06-11 05:04:13,502 INFO Script execution started

For custom alerting, integrate with system mail or APIs—never assume log files alone are enough.

Practical Example and Non-Obvious Tip

Sample utility: Check disk usage daily.
/opt/scripts/disk_usage.py:

#!/usr/bin/env python3
import subprocess

def get_usage():
    cp = subprocess.run(['df', '-hT'], capture_output=True, text=True)
    print(cp.stdout)

if __name__ == "__main__":
    get_usage()

Tip: Use -hT in df for human-readable output including filesystem type. Omit /proc and similar with custom filtering if noise becomes a problem.

Make it executable and assign to systemd timer—never miss low disk space again.

Wrap-up

The craft of executing and automating Python scripts on Linux depends on rigor: explicit interpreter selection, correct dependency isolation (venv), robust scheduling (cron or systemd), and disciplined logging.

There’s room for improvement—containerized execution, secrets management, integration with monitoring—but the patterns above form a solid baseline. Critically, adapt details to your distro’s nuances and operational requirements.

Known alternative: For even finer scheduling, third-party tools like at or specialized job runners (Airflow, Jenkins) may be justified in larger deployments.

Questions about specific scheduling edge cases or integrating with orchestration tools? Request deeper coverage as needed.