Choosing TDP Environment Size

This topic provides the "T-Shirt Sizing" for choosing the EnvironmentSize settings available on the Tetra Data Platform version 3.3.0. In this version of the software, there's significantly greater throughput of files processed per hour, up to 2.5x out-of-the-box. Contact Customer Support to explore optimizing your environment to increase performance further.

📘

NOTE

For information on T-Shirt sizing for TDP v3.2.x, see this topic.

The T-Shirt Sizing Table below is provided to help you understand the type and number of files that can be processed for a given "T-Shirt size," categorized as Small, Medium, or Large.

MetricDescriptionSmallMediumLarge
Data Lake Storage - Total Number of filesMax number of files in the data lake (RAW and Processed)<1 million<=3 million>3 million+
Workload - Heterogenous*Mix of all file types and sizes<=14K/hr<=22K/hr<=40K/hr
Workload - EmpowerEmpower Files (~ 5 MB)<=33K/hr<=61K/hr<=86K/hr
ReprocessingCreates a workflowId and put messages in queue for reprocessing1K batch/2Mins1K Batch/Min1K Batch/30 seconds

Throughput depends upon file types and sizes.

To generate this table, four main file types were chosen: Empower RAW JSON, AKTA RAW JSON, Binary, and CSV. Each was tested with different protocol versions and file sizes.

For Empower RAW JSON files, the protocol selected is common/empower-raw-to-ids:common/empower-raw-to-ids:v3.10.1, with the maximum file size tested of 1 GB binary, equivalent to 2.4 GB Empower RAW JSON uncompressed.

For the AKTA RAW JSON files, the protocol version is common/akta-raw-to-ids:v3.1.0, with maximum file size tested of 84 MB.

For Binary files, two protocol versions were selected. For FCS, common/bd-flow-cytometers-raw-to-ids:v1.1.2 is used, with a maximum file size tested of 141 MB. For CZI, common/extract-to-decorate:v2.1.0 common/zeiss-microscope-raw-to-ids:v0.2.4 is used, with a maximum file size tested of 12 GB.

For CSV files, common/perkinelmer-envision-raw-to-ids:v2.6.0 is the protocol version, with a maximum file size tested of 400 KB.

📘

NOTE

If you will be reprocessing a large number of files, consider going to a higher T-Shirt size.

For more details about instance sizing, please contact Customer Support.