API DocumentationProduct Documentation
Product Documentation

Agent Deployment Options

Suggest Edits

This topic shows the different Tetra Data Platform (TDP) upload methods and connection types available for Tetra Agents. Your organizationā€™s networking requirements determine the Agent connection type and upload method that you use.

šŸš§

IMPORTANT

For each deployment option's networking requirements, see Tetra Agent Allow List Endpoints.

Upload Methods

Agents can upload data to the TDP either directly through AWS API endpoints (S3 Direct Upload), or by using the TetraScience API (API).

šŸ“˜

NOTE

For performance reasons, the S3 Direct Upload option is recommended for most use casesā€”even when itā€™s not strictly required by file size. When an Agent uses the TDP (No Connector) configuration with the S3 Direct Upload option activated, the Agent calculates checksum and verifies it by using AWS 128-bit MD5 checksum. For Agents not using the S3 Direct Upload option, the Agent verifies the response code to determine if the upload was successful. Checksum isn't used.

S3 Direct Upload (Recommended)

The S3 Direct Upload option allows Agents to directly upload files to Amazon Simple Storage Service (Amazon S3) buckets in the TDP by using AWS API endpoints.

Hereā€™s how it works:

  1. The Agent connects to the TDP either through the internet (TDP (No Connector)) or an on-premises Tetra Hub.
  2. The Agent sends the TDP an Agent ID and, when not using a Tetra Data Hub, a JSON Web Token (JWT), by using the TetraScience API.
  3. The TDP responds with temporary AWS credentials that are scoped to the organizationā€™s resources.
  4. The Agent uses the temporary credentials it receives from the TDP to upload data directly to Amazon S3 by using AWS API endpoints.

šŸ“˜

NOTE

If the connection is set up through a Tetra Hub (which has a built-in L7 proxy) or a Data Hub with an L7 Proxy configured, the traffic goes through the proxy to reach AWS. For more information, see On-Premises Tetra Hub Connections.

Prerequisites

Setup

To have an Agent directly upload files to an Amazon S3 bucket in the TDP, set the S3 Direct Upload option to Yes when you configure the Agent.

For more information, see the documentation for the Agent that youā€™re installing (see Supported Tetra Agents).

šŸ“˜

NOTE

To not use the S3 Direct Upload option and have Agents upload data by using the TetraScience API only, set the S3 Direct Upload option to No. This is not recommended for most use cases, unless an Agentā€™s Tetra Hub doesnā€™t have access to AWS services.

Agent Connection Types

Agents require network access to TDP and AWS endpoints. There are two connection options for these endpoints:

  • TDP (No Connector): Connects Agents directly to the TDP and AWS APIs, or through a proxy that you configure in the Agent host server's settings.
  • Tetra Hub: Uses an on-premises Tetra Hub to proxy connections to TDP and AWS APIs.

TDP (No Connector) Connections

If the Tetra Agent is installed on a host server that has internet access either directly or through a proxy, you can configure it to upload files directly to the TDP.

Hereā€™s how it works:

  1. The Agent connects to the TDP either directly or or through a proxy configured in the Agent host server's settings.
  2. The Agent either uploads the files directly to the TDP through AWS API endpoints (S3 Direct Upload) or by using the TetraScience API.

šŸ“˜

NOTE

If the Agent host server requires a proxy to reach the required TDP and AWS endpoints, you can configure that proxy on the Agent host by using Windows settings. For more information, see Use a proxy server in Windows in the Microsoft documentation.

Prerequisites

Setup

To have an Agent upload files directly to the TDP through a TDP (No Connector) connection, do the following:

  1. Sign in to the TDP.
  2. Create an Agent entry in the TDP by following the instructions in Create a New Agent and choose No Connector.
  3. After creating the Agent, choose Continue Installing Agent to configure the installed Agent with connection information and service user credentials.

For more information, see the documentation for the Agent that youā€™re installing (see Supported Tetra Agents).

Tetra Hub Connections

šŸ“˜

NOTE

If the Tetra Agent has access to a proxy server that can reach the required TDP and AWS endpoints, configuring a Tetra Hub is not recommened. Instead, configure that proxy on the Agent host by using Windows settings. For more information, see Use a proxy server in Windows in the Microsoft documentation.

If the Tetra Agent is installed on a host server that otherwise doesn't have access to the required TDP and AWS endpoints (either directly or through another proxy), uploads can be sent to the TDP through an on-premises Tetra Hub. The Hub uses proxy servers to provide a secure connection to the TDP. You can also configure each Hubā€™s parent proxy server, if needed.

There are two Tetra Hub versions: Hub and Tetra Data Hub. Tetra Data Hub is on a deprecation track.

For new deployments, the latest Tetra Hub should be installed, except for the following use cases:

For more information, see Tetra Data Hub Deprecation.

šŸš§

IMPORTANT

After you decide what Tetra Hub connection option to use, you must specify that connection optionā€™s endpoint in the Agent Configuration section of the Agent Management Console when you configure the Agent. For more information, see the documentation for the Agent that youā€™re installing (see Supported Tetra Agents).

TDP Connection Endpoints for Tetra Hubs

  • Generic Data Connector (GDC): http://10.100.1.1:8888/generic-connector/v1/agent
  • Hub: http://10.100.1.2:8443/generic-connector/v1/agent
  • TDP: https://api.myorg-dev.com/v1/data-acquisition/agents

Hub Connections

For Agent connections set up through a Tetra Hub, uploads are sent by using the Hub itself as a pair of proxy servers.

Prerequisites

šŸš§

IMPORTANT

The Hub and its proxies must be online.

Setup

To have an Agent upload files to the TDP by using a Tetra Hub, do the following:

  1. Sign in to the TDP.
  2. In the left navigation menu, choose Data Sources. Then, choose Agents. The Agents page appears.
  3. Select the name of the Agent that you want to configure. A menu appears on the right.
  4. In the right menu, choose Install Agent. The Install Agent Locally page appears and displays a list of available Hubs.
  5. Select the Hub that you want to connect the Agent with. The Hubā€™s URL displays.
  6. Copy the Hubā€™s connection URL.
  7. Open the Agent Management Console. Then, navigate to the Agentā€™s configuration settings.
  8. In the Connection Url field, enter the Hubā€™s connection URL. The Agent validates the connection immediately and displays the updated connection status next to the Connection Url field.
  9. Choose Save. Then, choose Start to start the Agent and begin uploading files.

šŸ“˜

NOTE

Hub offers a reverse proxy to the TetraScience API, so Agents don't need to be registered with a particular Hub in the TDP. However, you must provide the Agent the connection URL of the Hub that itā€™s connecting to.

Data Hub Connections

šŸš§

IMPORTANT

Data Hub is on a deprecation track. For new deployments, Hub should be installed, except for the following use cases:

For more information, see Tetra Data Hub Deprecation.

For Agent connections set up through a Tetra Data Hub, uploads are sent to the TDP through a Generic Data Connector (GDC) and/or a proxy server (L7 Proxy Connector).

The L7 Proxy Connector allows Tetra Agents to securely reach AWS services that they otherwise donā€™t have direct access to (for example, Amazon S3). The GDC allows Agents without internet access to upload data directly to the Tetra Scientific Data and AI Cloud.

šŸ“˜

NOTE

For uploading large files (more than 500 MB), make sure that you use the S3 Direct Upload option along with both a GDC and L7 Proxy Connectors.

GDC Connections

Required when connecting an Agent to the TDP through a Data Hub, the Generic Data Connector (GDC) allows Agents without internet access to securely upload files to the Tetra Scientific Data and AI Cloud.

šŸ“˜

NOTE

To use the S3 Direct Upload option with a Data Hub, you must also set up an L7 Proxy Connector along with a GDC.

Hereā€™s how it works:

  1. The Agent connects to the GDC and sends it the Agent ID.
  2. The GDC responds with temporary credentials and the L7 Proxy Connectorā€™s port, if one is configured.
  3. The Agent uses the temporary credentials and proxy to upload files directly to the Tetra Scientific Data and AI Cloud through the GDC.

For more information, see Generic Data Connector.

Prerequisites

šŸš§

IMPORTANT

The Data Hub and its GDC (and L7C, if there is one) must be online.

Setup

To have an Agent upload files to the TDP by using a Tetra Data Hub, you must register the Agent on the TDP with a specific GDC.

To register a new Agent with a specific GDC, see Create a New Agent.

To change the GDC an existing Agent is registered with, see Change an Agentā€™s GDC or Connection Type.

L7 Proxy Connector

To use the S3 Direct Upload option with a Data Hub, you must add a Tetra L7 Proxy Connector (L7C) to your Hub along with a GDC.

šŸš§

IMPORTANT

For uploading large files (more than 500 MB), make sure that you select S3 Direct Upload in the Agent and point to the GDC. Without S3 Direct Upload configured on the Agent, large file uploads will fail, even if an L7C is configured.

Prerequisites

šŸš§

IMPORTANT

The Data Hub, its GDC, and the L7C must be online.

Setup

To create an L7 Proxy Connector and add it to a Data Hub, following the instructions in Create an L7C.

Manually Sync a Data Hub

If your Connector isn't working properly, didn't pick up the correct configuration, or if you've added an L7 Proxy Connector to your Data Hub, manually sync the Hub by doing the following:

  1. Sign in to the TDP.
  2. In the left navigation menu, choose Data Sources. Then, select Data Hub. The Data Management page appears.
  3. Find the Data Hub that you want to sync. Then, select the right hamburger menu icon in that Hub's row and choose Sync. It can take up to three minutes for the sync action to complete.

šŸš§

IMPORTANT

If youā€™ve made custom configuration changes within the Connector containersā€”for example, modifying an L7 proxyā€™s squid.conf fileā€”the changes will be lost during the synchronization process.

Updated 6 months ago