API DocumentationProduct Documentation

Agent Deployment Options

Suggest Edits

This topic shows the different Tetra Data Platform (TDP) upload methods and connection types available for Tetra Agents. Your organization’s networking requirements determine the Agent connection type and upload method that you use.

🚧

IMPORTANT

For each deployment option's networking requirements, see Tetra Agent Allow List Endpoints.

Upload Methods

Agents can upload data to the Tetra Scientific Data and AI Cloud either directly through AWS API endpoints (S3 Direct Upload), or by using the TetraScience API (API).

📘

NOTE

For performance reasons, the S3 Direct Upload option is recommended for most use cases—even when it’s not strictly required by file size. When an Agent uses the TDP (No Connector) configuration with the S3 Direct Upload option activated, the Agent calculates checksum and verifies it by using AWS 128-bit MD5 checksum. For Agents not using the S3 Direct Upload option, the Agent verifies the response code to determine if the upload was successful. Checksum isn't used.

S3 Direct Upload (Recommended)

The S3 Direct Upload option allows Agents to directly upload files to Amazon Simple Storage Service (Amazon S3) buckets in the Tetra Scientific Data and AI Cloud by using AWS API endpoints.

Here’s how it works:

  1. The Agent connects to the TDP either through the internet (TDP (No Connector)) or an on-premises Tetra Hub.
  2. The Agent sends the TDP an Agent ID and, when not using a Tetra Data Hub, a JSON Web Token (JWT), by using the TetraScience API.
  3. The TDP responds with temporary AWS credentials that are scoped to the organization’s resources.
  4. The Agent uses the temporary credentials it receives from the TDP to upload data directly to Amazon S3 by using AWS API endpoints.

📘

NOTE

If the connection is set up through a Tetra Hub (which has a built-in L7 proxy) or a Data Hub with an L7 Proxy configured, the traffic goes through the proxy to reach AWS. For more information, see On-Premises Tetra Hub Connections.

Prerequisites

Setup

To have an Agent directly upload files to an Amazon S3 bucket in the TDP, set the S3 Direct Upload option to Yes when you configure the Agent.

For more information, see the documentation for the Agent that you’re installing (see Supported Tetra Agents).

📘

NOTE

To not use the S3 Direct Upload option and have Agents upload data by using the TetraScience API only, set the S3 Direct Upload option to No. This is not recommended for most use cases, unless an Agent’s Tetra Hub doesn’t have access to AWS services.

Agent Connection Types

After a Tetra Agent extracts copies of data files from a data source, it can send those files to the Tetra Scientific Data and AI Cloud either through the internet (TDP (No Connector))or an on-premises Tetra Hub connection.

Internet Connections (TDP (No Connector))

If the Tetra Agent is installed on a host server that has internet access, you can configure it to upload files directly to the Tetra Scientific Data and AI Cloud

Here’s how it works:

  1. The Agent connects to the TDP through the internet.
  2. The Agent either uploads the files directly to the Tetra Scientific Data and AI Cloud through AWS API endpoints (S3 Direct Upload) or by using the TetraScience API.

Prerequisites

Setup

To have an Agent upload files directly to the Tetra Scientific Data and AI Cloud through an internet connection, do the following:

  1. Sign in to the TDP.
  2. Create an Agent entry in the TDP by following the instructions in Create a New Agent and choose No Connector.
  3. After creating the Agent, choose Continue Installing Agent to configure the installed Agent with connection information and service user credentials.

For more information, see the documentation for the Agent that you’re installing (see Supported Tetra Agents).

On-Premises Tetra Hub Connections

If the Tetra Agent is installed on a host server without internet access, uploads must be sent to the Tetra Scientific Data and AI Cloud through an on-premises Tetra Hub. The Hub uses proxy servers to provide a secure connection to the TDP. You can also configure each Hub’s parent proxy server, if needed.

There are two Tetra Hub versions: Hub and Tetra Data Hub. Tetra Data Hub is on a deprecation track.

For new deployments, Hub should be installed, except for the following use cases:

For more information, see Tetra Data Hub Deprecation.

🚧

IMPORTANT

After you decide what Tetra Hub connection option to use, you must specify that connection option’s endpoint in the Agent Configuration section of the Agent Management Console when you configure the Agent. For more information, see the documentation for the Agent that you’re installing (see Supported Tetra Agents).

TDP Connection Endpoints for Tetra Hubs

  • Generic Data Connector (GDC): http://10.100.1.1:8888/generic-connector/v1/agent
  • Hub: http://10.100.1.2:8443/generic-connector/v1/agent
  • TDP: https://api.myorg-dev.com/v1/data-acquisition/agents

Hub Connections

For Agent connections set up through a Tetra Hub, uploads are sent by using the Hub itself as a pair of proxy servers.

Prerequisites

🚧

IMPORTANT

The Hub and its proxies must be online.

Setup

To have an Agent upload files to the TDP by using a Hub, do the following:

  1. Sign in to the TDP.
  2. In the left navigation menu, choose Data Sources. Then, choose Agents. The Agents page appears.
  3. Select the name of the Agent that you want to configure. A menu appears on the right.
  4. In the right menu, choose Install Agent. The Install Agent Locally page appears and displays a list of available Hubs.
  5. Select the Hub that you want to connect the Agent with. The Hub’s URL displays.
  6. Copy the Hub’s connection URL.
  7. Open the Agent Management Console. Then, navigate to the Agent’s configuration settings.
  8. In the Connection Url field, enter the Hub’s connection URL. The Agent validates the connection immediately and displays the updated connection status next to the Connection Url field.
  9. Choose Save. Then, choose Start to start the Agent and begin uploading files.

📘

NOTE

Hub offers a reverse proxy to the TetraScience API, so Agents don't need to be registered with a particular Hub in the TDP. However, you must provide the Agent the connection URL of the Hub that it’s connecting to.

Data Hub Connections

🚧

IMPORTANT

Data Hub is on a deprecation track. For new deployments, Hub should be installed, except for the following use cases:

For more information, see Tetra Data Hub Deprecation.

For Agent connections set up through a Tetra Data Hub, uploads are sent to the TDP through a Generic Data Connector (GDC) and/or a proxy server (L7 Proxy Connector).

The L7 Proxy Connector allows Tetra Agents to securely reach AWS services that they otherwise don’t have direct access to (for example, Amazon S3). The GDC allows Agents without internet access to upload data directly to the Tetra Scientific Data and AI Cloud.

📘

NOTE

For uploading large files (more than 500 MB), make sure that you use the S3 Direct Upload option along with both a GDC and L7 Proxy Connectors.

GDC Connections

Required when connecting an Agent to the TDP through a Data Hub, the Generic Data Connector (GDC) allows Agents without internet access to securely upload files to the Tetra Scientific Data and AI Cloud.

📘

NOTE

To use the S3 Direct Upload option with a Data Hub, you must also set up an L7 Proxy Connector along with a GDC.

Here’s how it works:

  1. The Agent connects to the GDC and sends it the Agent ID.
  2. The GDC responds with temporary credentials and the L7 Proxy Connector’s port, if one is configured.
  3. The Agent uses the temporary credentials and proxy to upload files directly to the Tetra Scientific Data and AI Cloud through the GDC.

For more information, see Generic Data Connector.

Prerequisites

🚧

IMPORTANT

The Data Hub and its GDC (and L7C, if there is one) must be online.

Setup

To have an Agent upload files to the TDP by using a Data Hub, you must register the Agent on the TDP with a specific GDC.

To register a new Agent with a specific GDC, see Create a New Agent.

To change the GDC an existing Agent is registered with, see Change an Agent’s GDC or Connection Type.

L7 Proxy Connector

To use the S3 Direct Upload option with a Data Hub, you must add a Tetra L7 Proxy Connector (L7C) to your Hub along with a GDC.

🚧

IMPORTANT

For uploading large files (more than 500 MB), make sure that you select S3 Direct Upload in the Agent and point to the GDC. Without S3 Direct Upload configured on the Agent, large file uploads will fail, even if an L7C is configured.

Prerequisites

🚧

IMPORTANT

The Data Hub, its GDC, and the L7C must be online.

Setup

To create an L7 Proxy Connector and add it to a Data Hub, following the instructions in Create an L7C.

Manually Sync a Data Hub

If your Connector isn't working properly, didn't pick up the correct configuration, or if you've added an L7 Proxy Connector to your Data Hub, manually sync the Hub by doing the following:

  1. Sign in to the TDP.
  2. In the left navigation menu, choose Data Sources. Then, select Data Hub. The Data Management page appears.
  3. Find the Data Hub that you want to sync. Then, select the right hamburger menu icon in that Hub's row and choose Sync. It can take up to three minutes for the sync action to complete.

🚧

IMPORTANT

If you’ve made custom configuration changes within the Connector containers—for example, modifying an L7 proxy’s squid.conf file—the changes will be lost during the synchronization process.

Updated 14 days ago