Agent Deployment Options
This topic shows the different Tetra Data Platform (TDP) upload methods and connection types available for Tetra Agents. Your organizationās networking requirements determine the Agent connection type and upload method that you use.
IMPORTANT
For each deployment option's networking requirements, see Tetra Agent Allow List Endpoints.
Upload Methods
Agents can upload data to the TDP either directly through AWS API endpoints (S3 Direct Upload), or by using the TetraScience API (API).
NOTE
For performance reasons, the S3 Direct Upload option is recommended for most use casesāeven when itās not strictly required by file size. When an Agent uses the TDP (No Connector) configuration with the S3 Direct Upload option activated, the Agent calculates checksum and verifies it by using AWS 128-bit MD5 checksum. For Agents not using the S3 Direct Upload option, the Agent verifies the response code to determine if the upload was successful. Checksum isn't used.
S3 Direct Upload (Recommended)
The S3 Direct Upload option allows Agents to directly upload files to Amazon Simple Storage Service (Amazon S3) buckets in the TDP by using AWS API endpoints.
Hereās how it works:
- The Agent connects to the TDP either through the internet (TDP (No Connector)) or an on-premises Tetra Hub.
- The Agent sends the TDP an Agent ID and, when not using a Tetra Data Hub, a JSON Web Token (JWT), by using the TetraScience API.
- The TDP responds with temporary AWS credentials that are scoped to the organizationās resources.
- The Agent uses the temporary credentials it receives from the TDP to upload data directly to Amazon S3 by using AWS API endpoints.
NOTE
If the connection is set up through a Tetra Hub (which has a built-in L7 proxy) or a Data Hub with an L7 Proxy configured, the traffic goes through the proxy to reach AWS. For more information, see On-Premises Tetra Hub Connections.
Prerequisites
- A TDP organizational slug (orgSlug)
- A TDP user account within the organization
Setup
To have an Agent directly upload files to an Amazon S3 bucket in the TDP, set the S3 Direct Upload option to Yes when you configure the Agent.
For more information, see the documentation for the Agent that youāre installing (see Supported Tetra Agents).
NOTE
To not use the S3 Direct Upload option and have Agents upload data by using the TetraScience API only, set the S3 Direct Upload option to No. This is not recommended for most use cases, unless an Agentās Tetra Hub doesnāt have access to AWS services.
Agent Connection Types
Agents require network access to TDP and AWS endpoints. There are two connection options for these endpoints:
- TDP (No Connector): Connects Agents directly to the TDP and AWS APIs, or through a proxy that you configure in the Agent host server's settings.
- Tetra Hub: Uses an on-premises Tetra Hub to proxy connections to TDP and AWS APIs.
TDP (No Connector) Connections
If the Tetra Agent is installed on a host server that has internet access either directly or through a proxy, you can configure it to upload files directly to the TDP.
Hereās how it works:
- The Agent connects to the TDP either directly or or through a proxy configured in the Agent host server's settings.
- The Agent either uploads the files directly to the TDP through AWS API endpoints (S3 Direct Upload) or by using the TetraScience API.
NOTE
If the Agent host server requires a proxy to reach the required TDP and AWS endpoints, you can configure that proxy on the Agent host by using Windows settings. For more information, see Use a proxy server in Windows in the Microsoft documentation.
Prerequisites
- A TDP organizational slug (orgSlug)
- A TDP user account within the organization
- The required allow list endpoints must be added to your organizationās allow list, including the TetraScience API (
https://api.tetrascience.com/
)
Setup
To have an Agent upload files directly to the TDP through a TDP (No Connector) connection, do the following:
- Sign in to the TDP.
- Create an Agent entry in the TDP by following the instructions in Create a New Agent and choose No Connector.
- After creating the Agent, choose Continue Installing Agent to configure the installed Agent with connection information and service user credentials.
For more information, see the documentation for the Agent that youāre installing (see Supported Tetra Agents).
Tetra Hub Connections
NOTE
If the Tetra Agent has access to a proxy server that can reach the required TDP and AWS endpoints, configuring a Tetra Hub is not recommened. Instead, configure that proxy on the Agent host by using Windows settings. For more information, see Use a proxy server in Windows in the Microsoft documentation.
If the Tetra Agent is installed on a host server that otherwise doesn't have access to the required TDP and AWS endpoints (either directly or through another proxy), uploads can be sent to the TDP through an on-premises Tetra Hub. The Hub uses proxy servers to provide a secure connection to the TDP. You can also configure each Hubās parent proxy server, if needed.
There are two Tetra Hub versions: Hub and Tetra Data Hub. Tetra Data Hub is on a deprecation track.
For new deployments, the latest Tetra Hub should be installed, except for the following use cases:
- Tetra Solace Connector integrations
- Facilitating on-premises HTTP requests from pipelines if AWS PrivateLink or an API gateway doesn't meet your requirements
For more information, see Tetra Data Hub Deprecation.
IMPORTANT
After you decide what Tetra Hub connection option to use, you must specify that connection optionās endpoint in the Agent Configuration section of the Agent Management Console when you configure the Agent. For more information, see the documentation for the Agent that youāre installing (see Supported Tetra Agents).
TDP Connection Endpoints for Tetra Hubs
- Generic Data Connector (GDC):
http://10.100.1.1:8888/generic-connector/v1/agent
- Hub:
http://10.100.1.2:8443/generic-connector/v1/agent
- TDP:
https://api.myorg-dev.com/v1/data-acquisition/agents
Hub Connections
For Agent connections set up through a Tetra Hub, uploads are sent by using the Hub itself as a pair of proxy servers.
Prerequisites
- A Hub, created and installed
- The required Tetra Hub allow list endpoints must be added to your organizationās allow list
IMPORTANT
The Hub and its proxies must be online.
Setup
To have an Agent upload files to the TDP by using a Tetra Hub, do the following:
- Sign in to the TDP.
- In the left navigation menu, choose Data Sources. Then, choose Agents. The Agents page appears.
- Select the name of the Agent that you want to configure. A menu appears on the right.
- In the right menu, choose Install Agent. The Install Agent Locally page appears and displays a list of available Hubs.
- Select the Hub that you want to connect the Agent with. The Hubās URL displays.
- Copy the Hubās connection URL.
- Open the Agent Management Console. Then, navigate to the Agentās configuration settings.
- In the Connection Url field, enter the Hubās connection URL. The Agent validates the connection immediately and displays the updated connection status next to the Connection Url field.
- Choose Save. Then, choose Start to start the Agent and begin uploading files.
NOTE
Hub offers a reverse proxy to the TetraScience API, so Agents don't need to be registered with a particular Hub in the TDP. However, you must provide the Agent the connection URL of the Hub that itās connecting to.
Data Hub Connections
IMPORTANT
Data Hub is on a deprecation track. For new deployments, Hub should be installed, except for the following use cases:
- Tetra Solace Connector integrations
- Facilitating on-premises HTTP requests from pipelines if AWS PrivateLink or an API gateway doesn't meet your requirements
For more information, see Tetra Data Hub Deprecation.
For Agent connections set up through a Tetra Data Hub, uploads are sent to the TDP through a Generic Data Connector (GDC) and/or a proxy server (L7 Proxy Connector).
The L7 Proxy Connector allows Tetra Agents to securely reach AWS services that they otherwise donāt have direct access to (for example, Amazon S3). The GDC allows Agents without internet access to upload data directly to the Tetra Scientific Data and AI Cloud.
NOTE
For uploading large files (more than 500 MB), make sure that you use the S3 Direct Upload option along with both a GDC and L7 Proxy Connectors.
GDC Connections
Required when connecting an Agent to the TDP through a Data Hub, the Generic Data Connector (GDC) allows Agents without internet access to securely upload files to the Tetra Scientific Data and AI Cloud.
NOTE
To use the S3 Direct Upload option with a Data Hub, you must also set up an L7 Proxy Connector along with a GDC.
Hereās how it works:
- The Agent connects to the GDC and sends it the Agent ID.
- The GDC responds with temporary credentials and the L7 Proxy Connectorās port, if one is configured.
- The Agent uses the temporary credentials and proxy to upload files directly to the Tetra Scientific Data and AI Cloud through the GDC.
For more information, see Generic Data Connector.
Prerequisites
- A Data Hub, created and installed
- A GDC, added to the Data Hub
- The required Tetra Data Hub allow list endpoints must be added to your organizationās allow list
IMPORTANT
The Data Hub and its GDC (and L7C, if there is one) must be online.
Setup
To have an Agent upload files to the TDP by using a Tetra Data Hub, you must register the Agent on the TDP with a specific GDC.
To register a new Agent with a specific GDC, see Create a New Agent.
To change the GDC an existing Agent is registered with, see Change an Agentās GDC or Connection Type.
L7 Proxy Connector
To use the S3 Direct Upload option with a Data Hub, you must add a Tetra L7 Proxy Connector (L7C) to your Hub along with a GDC.
IMPORTANT
For uploading large files (more than 500 MB), make sure that you select S3 Direct Upload in the Agent and point to the GDC. Without S3 Direct Upload configured on the Agent, large file uploads will fail, even if an L7C is configured.
Prerequisites
- A Data Hub, created and installed
- A GDC, added to the Data Hub
- The required Tetra Data Hub allow list endpoints must be added to your organizationās allow list
IMPORTANT
The Data Hub, its GDC, and the L7C must be online.
Setup
To create an L7 Proxy Connector and add it to a Data Hub, following the instructions in Create an L7C.
Manually Sync a Data Hub
If your Connector isn't working properly, didn't pick up the correct configuration, or if you've added an L7 Proxy Connector to your Data Hub, manually sync the Hub by doing the following:
- Sign in to the TDP.
- In the left navigation menu, choose Data Sources. Then, select Data Hub. The Data Management page appears.
- Find the Data Hub that you want to sync. Then, select the right hamburger menu icon in that Hub's row and choose Sync. It can take up to three minutes for the sync action to complete.
IMPORTANT
If youāve made custom configuration changes within the Connector containersāfor example, modifying an L7 proxyās
squid.conf
fileāthe changes will be lost during the synchronization process.
Updated 5 months ago