Automate FTP Uploads Every Night with Cron and Python Script

You finish work, close the laptop, and assume the evening backup will happen. By morning the files sit exactly where you left them. Manual FTP uploads fail the moment you forget, and forgetting is inevitable when the task repeats every single day.

A cron job paired with a short Python script removes the human step. The server wakes at the hour you choose, logs into the remote host, pushes the files, and logs the outcome. No desktop app running, no reminder alarm, no weekend guilt.

Why scheduled FTP still matters in a cloud-first world

Cloud APIs dominate the conversation around file transfer workflows, yet thousands of legacy systems, shared hosting panels, and compliance-locked environments still speak only FTP or SFTP. A municipal records office may mandate nightly submission to an FTP drop folder. A manufacturing partner may refuse to open firewall ports for anything newer. The protocol is old; the need is current.

Cron exists on every Unix-like system by default. Python's ftplib ships in the standard library. Together they form a zero-cost automation stack that runs without third-party services, subscription fees, or vendor lock-in. When the upload window is predictable and the destination fixed, a cron job is simpler than orchestrating a cloud function with API keys and retry logic.

Manual uploads also introduce timing drift. You might run the transfer at 9 PM one night, 11 PM the next, or skip Friday entirely. Downstream processes that expect files by a certain hour break silently. A cron entry enforces the schedule with operating-system reliability.

Build a basic ftplib upload script in under fifty lines

Start with a Python file named ftp_upload.py. Import ftplib, os, and datetime. Define four variables at the top: FTP_HOST, FTP_USER, FTP_PASS, and LOCAL_DIR. Hard-code them for now; environment variables come later.

Open an FTP connection inside a try block. Use FTP_TLS if the server supports explicit TLS; fall back to plain FTP only when encryption is unavailable. Call login with username and password, then cwd to the target remote directory. Loop over os.listdir(LOCAL_DIR), open each file in binary mode, and pass the handle to storbinary with the STOR command. Close the connection in a finally block.

Add a timestamp print statement before and after the loop so logs show start time, file count, and finish time. Wrap the entire script in if __name__ == '__main__' to keep it importable. Test the script manually from the command line before scheduling it. If it fails, you will see the traceback immediately instead of hunting through cron mail twelve hours later.

Handle errors without crashing the whole job

A missing file, a dropped connection, or a wrong password will raise an exception. If the script exits with status one, cron will email root (or the user) with the traceback, but the next night's job still runs. That behavior is acceptable for non-critical backups. For mission-critical transfers, catch ftplib.all_errors and log the message to a file instead of letting it bubble up.

Create a simple append-only log in /var/log/ftp_upload.log or inside the project directory if /var/log requires root. Write one line per run: ISO timestamp, success or failure, file count, and any error class name. Rotate the log weekly with logrotate or a shell script so it does not grow without bound.

Consider a retry loop for transient network errors. Wrap the FTP block in a for attempt in range(3) loop with a sixty-second sleep between tries. If all three attempts fail, log the failure and exit zero so cron does not send mail. This pattern prevents a single blip from generating alarm fatigue while still recording the incident for morning review.

Write the crontab entry and choose the right minute

Run crontab -e to open your user crontab in the default editor. Each line has five time fields followed by the command. Minute (0-59), hour (0-23), day of month (1-31), month (1-12), day of week (0-7, where both 0 and 7 mean Sunday). An asterisk means every value.

To run at 2:30 AM every day, write 30 2 * * * /usr/bin/python3 /home/user/ftp_upload.py. Use absolute paths for both the interpreter and the script because cron runs with a minimal PATH. Redirect stdout and stderr to a file if you want to capture print statements: append >> /home/user/ftp_upload.out 2>&1 to the end of the line.

Pick an hour when the source files are stable and the network is quiet. Avoid the top of the hour if dozens of other cron jobs fire then; stagger by a few minutes to spread load. If the upload takes longer than twenty-four hours to complete, add flock or a PID file check at the start of the script so a second instance does not launch while the first is still running.

Secure credentials with environment variables and restricted permissions

Hard-coded passwords in the script are visible to anyone with read access and will appear in process listings while the script runs. Move FTP_USER and FTP_PASS into environment variables. Create a file named .env in the same directory as the script, write FTP_USER=yourname and FTP_PASS=secret on separate lines, then chmod 600 .env so only the owner can read it.

Install python-dotenv with pip, import load_dotenv at the top of the script, and call load_dotenv() before reading os.getenv('FTP_USER'). The cron environment is sparse, so source the .env file in the crontab line itself: 30 2 * * * export $(cat /home/user/.env | xargs) && /usr/bin/python3 /home/user/ftp_upload.py. This pattern keeps secrets out of the script and out of the global environment.

For production systems, use a secret manager or SSH key authentication if the server supports SFTP. Paramiko replaces ftplib for SFTP and accepts a private-key path instead of a password. The key file should also be chmod 600 and owned by the cron user.

Monitor success and catch silent failures

Cron will email output only if the job writes to stderr or exits non-zero. A script that silently fails to upload files because the remote directory was renamed will produce no alert. Add a final check that compares the count of files sent against the count of files in LOCAL_DIR. If they do not match, write a warning to the log and exit one.

For higher assurance, write a second script that runs an hour after the upload job and queries the FTP server for a directory listing. Compare modification timestamps or file sizes against the local copies. If a discrepancy appears, send an email using smtplib or post a message to a Slack webhook. This watchdog pattern catches both upload failures and cases where the server accepted the file but silently corrupted it.

Log rotation and alerting are easier when the log format is machine-readable. Use JSON lines (one JSON object per line) instead of free-form text. Each entry includes timestamp, status, file_count, duration_seconds, and error_message. A monitoring script can parse the log with one line of Python per record and trigger alerts based on status or missing runs.

Key facts

• ftplib ships with Python's standard library; no pip install required.
• Cron reads /etc/crontab and per-user tables in /var/spool/cron/crontabs on most Linux systems.
• A cron line with five asterisks runs every minute; test jobs often use */5 for every five minutes.
• FTP_TLS.prot_p() must be called after login to encrypt the data channel, the control channel.

Can I run the same script on Windows without cron?

Yes. Use Task Scheduler to launch pythonw.exe with the script path as an argument. Set the trigger to daily at your chosen time and configure the task to run whether the user is logged on or not. The script itself does not need changes.

What happens if the FTP server is down when cron fires?

The script will raise a socket timeout or connection-refused error. If you catch the exception and exit zero, cron will not send mail, but the files remain local. A retry loop or a second cron entry thirty minutes later can recover from brief outages.

How do I upload only files modified in the last twenty-four hours?

Before calling storbinary, check os.path.getmtime(filepath). Compare it to time.time() minus 86400 seconds. Skip files older than that threshold. This filter prevents re-uploading static files every night and reduces transfer time.

Related reading: What we do · About us