Data Backup

Datalake files:

  • The S3 datalake bucket has versioning enabled and delete is implemented as soft delete: actual file content is never removed, we just add delete markers to the files.
    Additionally, the buckets will not be removed in case TDP is accidentally uninstalled from CloudFormation or ServiceCatalog.

RDS Database:

  • Custom backups are by default performed once per day, with a retention period of 30 days. Frequency and retention can be modified using the deployment parameters. There is a default AWS limit of having a maximum of 100 snapshots per DB instance, but it can be increased via AWS support.
  • AWS automated backups - performed daily with a retention period of 7 days


  • Backed up 4 times per day in an S3 bucket. Data in ElasticSearch can also be recreated from S3, using an existing ECS re-processing task