Exporting Amazon RDS Data to S3

Exporting Amazon RDS Data to S3

Reading time1 min
#Cloud#AWS#Database#RDS#S3#Automation

Exporting Amazon RDS Data to S3

"Export RDS to S3" covers several different needs: analytics on a copy of production data, archiving old rows, a portable backup that can be restored elsewhere, or a continuous feed into a data lake. Each has a different tool, and picking the wrong one is the main source of trouble. In particular, snapshot exports are not backups: you cannot restore a database from them.

Which method for which goal

GoalMethodEngines
Analytics, data lake, AthenaSnapshot export (Parquet)RDS for MySQL, MariaDB, PostgreSQL; Aurora MySQL and PostgreSQL
Export a query result or archive rowsaws_s3.query_export_to_s3RDS for PostgreSQL, Aurora PostgreSQL
Export a query resultSELECT ... INTO OUTFILE S3Aurora MySQL only
Restorable logical backupmysqldump or pg_dump streamed to S3All MySQL, MariaDB, PostgreSQL
Native backup filesNative backup to S3RDS for SQL Server, RDS for Oracle (Data Pump)
Continuous change feedAWS DMS with an S3 targetMost engines

Snapshot export to S3

RDS restores the snapshot in the background, extracts the data and writes it to S3 as compressed Parquet, partitioned per table. It does not touch the running instance, so there is no load on production. You can export all data or specific databases, schemas and tables. Supported engine versions and regions vary, so check the RDS documentation for yours.

Prerequisites:

  1. An S3 bucket in the same region as the snapshot.
  2. An IAM role that export.rds.amazonaws.com can assume, with s3:PutObject*, s3:GetObject*, s3:ListBucket, s3:DeleteObject* and s3:GetBucketLocation on that bucket.
  3. A symmetric KMS key. The export is always encrypted with it, and the key policy must allow kms:CreateGrant and kms:DescribeKey for the caller.
aws iam create-role --role-name rds-s3-export-role \
  --assume-role-policy-document '{
    "Version": "2012-10-17",
    "Statement": [{
      "Effect": "Allow",
      "Principal": { "Service": "export.rds.amazonaws.com" },
      "Action": "sts:AssumeRole"
    }]
  }'

Attach a policy with the S3 actions above to this role, then start the export:

aws rds start-export-task \
  --export-task-identifier my-db-export-2026-10-06 \
  --source-arn arn:aws:rds:eu-central-1:123456789012:snapshot:rds:my-db-2026-10-06-02-10 \
  --s3-bucket-name my-db-exports \
  --s3-prefix my-db \
  --iam-role-arn arn:aws:iam::123456789012:role/rds-s3-export-role \
  --kms-key-id arn:aws:kms:eu-central-1:123456789012:key/1234abcd-12ab-34cd-56ef-1234567890ab \
  --export-only app.orders app.customers

aws rds describe-export-tasks --export-task-identifier my-db-export-2026-10-06

Leave out --export-only to export everything. Export task identifiers must be unique, so put a date in them. For Aurora, --source-arn can also be a DB cluster ARN, which exports directly from the cluster without taking a snapshot first.

Things to know:

  • Exports take a while: the task first restores the snapshot, then extracts. Tables without a suitable numeric key are exported single-threaded and are slow.
  • At most five export tasks run concurrently per account.
  • Views are not exported. Tables with unsupported characters in column names, and rows close to 2 GB, are skipped.
  • To query the result, run an AWS Glue crawler on the prefix and use Athena or Redshift Spectrum.

Query export from PostgreSQL

RDS for PostgreSQL and Aurora PostgreSQL can write a query result straight to S3 with the aws_s3 extension. Attach a role to the instance for the s3Export feature first:

aws rds add-role-to-db-instance --db-instance-identifier my-pg \
  --feature-name s3Export \
  --role-arn arn:aws:iam::123456789012:role/rds-pg-s3-export

(For Aurora, use add-role-to-db-cluster.) Then in the database:

CREATE EXTENSION IF NOT EXISTS aws_s3 CASCADE;

SELECT * FROM aws_s3.query_export_to_s3(
  'SELECT * FROM orders WHERE created_at < now() - interval ''1 year''',
  aws_commons.create_s3_uri('my-db-exports', 'archive/orders-2025.csv', 'eu-central-1'),
  options := 'format csv, header true'
);

This is the simplest way to archive old rows: export, verify the file, then delete the rows in batches.

Query export from Aurora MySQL

Aurora MySQL supports SELECT ... INTO OUTFILE S3 once an IAM role is associated with the cluster and set in the aws_default_s3_role cluster parameter:

SELECT * FROM orders WHERE created_at < '2025-01-01'
INTO OUTFILE S3 's3-eu-central-1://my-db-exports/archive/orders'
FIELDS TERMINATED BY ',' LINES TERMINATED BY '\n'
MANIFEST ON;

On Aurora MySQL version 3 the database user also needs the AWS_SELECT_S3_ACCESS role. RDS for MySQL and MariaDB do not support this statement.

Restorable dumps streamed to S3

When you need something you can load into another MySQL or PostgreSQL server, use the engine's own dump tool from a host that can reach the database (an EC2 instance, an ECS task, a CI runner) and stream the output to S3 without a local file:

mysqldump -h my-db.abcdefgh1234.eu-central-1.rds.amazonaws.com -u backup -p"$DB_PASSWORD" \
  --single-transaction --routines --triggers app \
  | gzip | aws s3 cp - s3://my-db-exports/dumps/app-$(date +%F).sql.gz --expected-size 100000000000

pg_dump -h my-pg.abcdefgh1234.eu-central-1.rds.amazonaws.com -U backup -Fc app \
  | aws s3 cp - s3://my-db-exports/dumps/app-$(date +%F).dump

--expected-size (in bytes) is needed for streams larger than 50 GB so the CLI picks a large enough multipart chunk size. Use set -o pipefail in scripts so a failed dump does not upload a truncated file. For RDS for SQL Server, native .bak backups to S3 go through the SQLSERVER_BACKUP_RESTORE option and msdb.dbo.rds_backup_database. For RDS for Oracle, Data Pump files are copied to S3 with the S3_INTEGRATION option.

Continuous export with AWS DMS

If the goal is a data lake that stays current, use AWS DMS with the RDS instance as source and S3 as target. It does a full load and then writes changes (CDC) as CSV or Parquet files. That needs binary logging (MySQL) or logical replication (PostgreSQL) on the source, plus a DMS replication instance or DMS Serverless.

Automate snapshot exports

A common setup exports the latest automated snapshot once a day. An EventBridge Scheduler schedule (or an EventBridge rule on RDS snapshot events) invokes a Lambda function:

import os
from datetime import datetime, timezone

import boto3

rds = boto3.client("rds")


def lambda_handler(event, context):
    snapshots = rds.describe_db_snapshots(
        DBInstanceIdentifier=os.environ["DB_INSTANCE"], SnapshotType="automated"
    )["DBSnapshots"]
    latest = max(
        (s for s in snapshots if s["Status"] == "available"),
        key=lambda s: s["SnapshotCreateTime"],
    )
    task_id = f"{os.environ['DB_INSTANCE']}-{datetime.now(timezone.utc):%Y%m%d%H%M}"
    rds.start_export_task(
        ExportTaskIdentifier=task_id,
        SourceArn=latest["DBSnapshotArn"],
        S3BucketName=os.environ["BUCKET"],
        S3Prefix=os.environ["DB_INSTANCE"],
        IamRoleArn=os.environ["EXPORT_ROLE_ARN"],
        KmsKeyId=os.environ["KMS_KEY_ARN"],
    )
    return {"export_task": task_id, "snapshot": latest["DBSnapshotIdentifier"]}

The function's role needs rds:DescribeDBSnapshots, rds:StartExportTask, iam:PassRole on the export role, and kms:DescribeKey and kms:CreateGrant on the key. Add an S3 lifecycle rule on the export prefix so old exports expire or move to a colder storage class.

AWS Data Pipeline, which older guides use for this, is closed to new customers. Use EventBridge with Lambda, or Step Functions when the workflow has several steps (export, wait, crawl, notify).

Quick reference

# Snapshot to Parquet in S3
aws rds start-export-task --export-task-identifier ID --source-arn SNAPSHOT_ARN \
  --s3-bucket-name BUCKET --iam-role-arn ROLE_ARN --kms-key-id KEY_ARN
aws rds describe-export-tasks --export-task-identifier ID

# PostgreSQL query to S3
aws rds add-role-to-db-instance --db-instance-identifier DB --feature-name s3Export --role-arn ROLE_ARN
# SELECT * FROM aws_s3.query_export_to_s3('query', aws_commons.create_s3_uri('bucket','key','region'));

# Restorable dump
mysqldump --single-transaction DB | gzip | aws s3 cp - s3://BUCKET/dump.sql.gz