Select TDP Deployment Size for TDP 3.4.x
This topic provides the "T-Shirt Sizing" for choosing the EnvironmentSize settings available for the Tetra Data Platform version 3.4.x. There have been significant performance increases in this version of TDP. Contact Customer Support if you want to explore optimizing your deployment to increase performance further.
NOTE
For information on T-Shirt sizing for TDP v3.3.x, see this topic.
The T-Shirt Sizing Table below is provided to help you understand the type and number of files that can be processed for a given "T-Shirt size," categorized as Small, Medium, or Large. Custom sizes are possible based on various workloads. Customers can work with TetraScience to review their Platform configurations.
| Description | Small* | Medium* | Large* | |
|---|---|---|---|---|
| Capacity (Total Number of files) | Maximum number of files in the data lake. | ~3 Million Files | ~5 Million Files | ~5+ Million Files |
| Size of files | Storage Requirement (RDS and ES) | Platform Storage requirement depends on file sizes. | Platform Storage requirement will depend on file sizes | Platform Storage requirement depends on file sizes. |
| Variable workload | Workload consists of different file types/sizes. | ~26k/hr | ~45k/hr | ~48k/hr |
| Single file type/size (~ 5 MB) | Includes single file type/size (Empower 5MB) a customer can upload. | ~55k/hr | ~90k/hr | ~200k/hr |
| Reprocessing | Creates a workflowId and put messages in queue for reprocessing. | 1k batch/2Mins | 1k Batch/Min | 1k Batch/30 seconds |
* Custom sizes are possible based on various workloads. Customers can work with TetraScience to review their platform configurations.
NOTE
If you will be reprocessing a large number of files, consider going to a higher T-Shirt size.
To generate this table, four main file types were chosen: Empower RAW JSON, AKTA RAW JSON, Binary, and CSV. Each was tested with different protocol versions and file sizes:
-
For Empower RAW JSON files, the protocol selected is common/empower-raw-to-ids/common/empower-raw-to-ids, with the maximum file size tested of 1 GB binary, equivalent to 2.4 GB Empower RAW JSON uncompressed.
-
For the AKTA RAW JSON files, the protocol version is common/akta-raw-to-ids, with maximum file size tested of 84 MB.
-
For Binary files, two protocol versions were selected. For FCS, common/bd-flow-cytometers-raw-to-ids is used, with a maximum file size tested of 141 MB. For CZI, common/extract-to-decorate common/zeiss-microscope-raw-to-ids is used, with a maximum file size tested of 12 GB.
-
For CSV files, common/perkinelmer-envision-raw-to-ids is the protocol version, with a maximum file size tested of 400 KB.
For more details about instance sizing, please contact Customer Support.
Updated 2 months ago