API DocumentationProduct Documentation
Product Documentation

Self-Service Tetra Data Pipelines

Suggest Edits

TetraScience provides many pre-built pipelines that can help you create queryable, harmonized Tetra Data, and then enrich and push that data to downstream systems. To extend the capabilities of these pipelines and the Tetra Data Platform (TDP), you can also create your own custom self-service Tetra Data pipelines (SSPs).

For example, you can use SSPs to do any of the following:

  • Create new parsers for scientific instrument data.
  • Add labels to your data to make it findable, and then discover use patterns.
  • Enrich your data by combining it with other third-party data on the TDP.
  • Automate the manual work of data preprocessing for analytics software or other proprietary processes.
  • Send processed data to other applications, such as machine learning tools, electronic lab notebooks (ELNs), or laboratory information management systems (LIMS).

For more information about example SSP use cases, see Example Use Cases for SSPs.

SSP Architecture

The following diagram shows an example SSP workflow:

SSP architecture diagram

SSP architecture diagram

The diagram shows the following workflow:

  1. Task scripts, which are the building blocks of protocols, contain the code for the business logic needed to process data. These task scripts are written in Python.
  2. Protocols are written in a YAML file (protocol.yml), which specifies configuration elements and outlines the execution order of task script functions.
  3. Task scripts and protocols are deployed to the TDP by using the TetraScience Software Development Kit (SDK) 2.0.
  4. Once these artifacts are on the TDP, you can create an SSP by doing the following:
    • Specifying trigger conditions
    • Providing configuration values
    • Leveraging the protocol that you’ve created by using either commonly available task scripts, or ones you’ve created yourself

🚧

IMPORTANT

All files uploaded by SSPs must be less than 50 MB.

CI/CD Workflows for SSPs

To help manage your SSPs, you can build your own continuous integration and continuous delivery (CI/CD) process by using whatever source code version control system you want.

The following diagram shows an example GitHub-based CI/CD workflow for managing SSPs:

SSP CI/CD workflow example

Create SSPs

To learn more about how to create your own SSPs, see the following:

Updated 5 months ago

What’s Next