After the Tetra Agent extracts copies of the data files from the data source, it sends those files to the Tetra Data Platform (TDP). This page shows how to configure integrations and connectors to transfer data to the TDP.

Tetra Agent Integration and Connector Overview

TetraScience provides several different ways for Tetra Agents to integrate with the TDP.

Ways that Tetra Agents Can Integrate with TDP

Selecting a Connector: GDC or TDP

To determine which type of connector to use, you need to know two things:

• Is the Tetra Agent installed on a machine that has internet access?
• Are the files to be transferred to the data lake large (more than 500 MB) or small (less than 500 MB)?

Connector Decision Table

• If the agent has internet access, use TDP (No Connector), and make sure that you have the S3 Direct Upload configuration enabled.
• If the agent has internet access, and files are small, you can use No Connector without S3 Direct Upload.
• If the agent does not have internet access, and the files are large, use the Generic Data Connector (GDC), which should be on the Data Hub. You will also need to use the L7 Proxy Connector (L7C). The L7C is a simple proxy server that uploads files directly to the data lake. L7C accommodates large files. The L7 Proxy Connector should also be set up on the Data Hub. Make sure that you have the S3 Direct Upload configuration enabled.
• If the agent does not have internet access, you cannot configure an L7C. If the files are small, you can use the Generic Data Connector (GDC) on the DataHub without S3 Direct Upload.

📘

NOTE

Typically, the S3 Direct Upload is preferred—even in cases where it’s not strictly required by file size— because of its performance.

• The Agent's host machine must be on Windows 10 or newer, or Windows Server 2016 for the Cloud Data Connector to connect to it properly.

Connection URL

After determining which connector to use, you must specify the Connection URL. The following are example GDC and TDP connection URLs:

• GDC URL example: http://10.100.1.1:8888/generic-connector/v1/agent
• TDP URL example: https://api.myorg-dev.com/v1/data-acquisition/agents

Connector/Integration Configuration Instructions

Generic Data Connector (GDC)

Tetra Agents can be integrated with the Tetra Data Platform through the Tetra Data Hub. The Tetra Data Hub is an on-premise server that facilitates securely transferring data extracted from a Tetra Agent to the Tetra Data Platform. The Agent connects to the Data Hub through a data connector.

Prerequisites

• A Tetra Data Hub must be set up.
• A connector must be added to the Tetra Data Hub.
• The Data Hub and Data Connector must be online.

Procedures

Because the Tetra Agent transfers data with a Tetra Data Hub Connector, you must register the Tetra Agent first on the TDP with a specific connector.

To add a Tetra Agent to your existing connector on the TDP, do the following:

1. Sign in to the TDP.
2. In the left navigation bar, choose the hamburger menu. Then, choose Data Sources. A drop-down list appears.
3. Choose Data Hub.
4. On the Data Management page, find the connector that you want to add the Tetra Agent to. Select the menu icon, and then choose Edit.
5. Scroll to the Agent section. Then, choose Add Agent.
6. The Agent ID is autogenerated by the TDP. You just need to give the Agent a name. Optionally, you can also specify Agent metadata and tags for quick identification on the Data Management page.
7. Choose Save.
8. The new Agent appears with the connector on the Data Management page. The status of the Agent changes to online when the Agent service starts and successfully connects to the connector.

Using the "TDP (No Connector)" Option (TDP)

Besides the connection through the Data Hub/Data Connector, the Agent can directly upload the files to the TDP.

Prerequisites

• Client Organization Slug (orgSlug) is created in the TDP.
• A user account is created in the Organization Slug

Procedures

1. Sign in to the TDP.
2. Create an Agent entry in the TDP by doing the following:
   a. In the left navigation bar, choose the hamburger menu. Then, choose Select Data Sources. A drop-down list appears.
   b. Choose Agents.
   c. Choose Create New Agent. The Add Agent dialog appears.
   d. In the Add Agent dialog, choose No Connector. Then, choose Next.
   e. Enter the name, description, and Agent type. If you need to add an attribute (for example, a label), choose the Add/Edit Attributes button.
   f. Choose Create Agent. The Agent is created in TDP.
   g. Choose Install Agent Now.
   h. Copy the Agent ID and Connection URL to your clipboard. In your Agent, enter those values in Agent ID and Connection URL text fields, respectively, and add your organization in the Org Slug text field.
3. Create an Authorization token for a Service User. For instructions, see Generate a JSON Web Token for an Existing Service User.
4. Copy the authorization token into the Authorization field on your Agent. A Connected message appears next to your Connection URL.
5. Enter an Integration Name. Then, choose Create.
6. Provide the following detailed information.:
   Note: Agent Id is generated by default.
   a. Select aSource Type. Source Type is mandatory. The default setting is unknown.
   b. Enter a Name and Description.
   c. Enter metadata and tags.
   d. Choose Finish to create the Agent. The automatically generated Agent Id will be used in the Agent configuration.
7. Use the Agent ID and JSON Token to set up the Agent to upload the files to the TDP.

L7 Proxy Connector (L7C)

It is optional to create an L7 Proxy Connector (L7C) along with Generic Data Connector (GDC). The purpose of an L7 Proxy Connector is to allow an end-user to upload large files (size is larger than 500 MB) to the TDP by using the Generic Data Connector option.

❗️

For large files (more than 500MB)

Remember to select S3 Direct Upload in the Agent and point to the L7 Proxy connector. L7 Proxy without S3 Direct Upload configured will fail at uploading large files.

To create an L7 Proxy Connector, do the following:

1. Sign in to the TDP.
2. In the left navigation bar, choose the hamburger menu. Then, choose Data Hub.
3. Choose Add Connector. A drop-down list appears.
4. Choose L7 Proxy Connector.
5. Enter the L7C's Name. Then, enter the Port number. The default port is 3128.
   Note: Make sure that you check with your system administrator to make sure that this port is available.
6. Choose Save, and then wait a couple of minutes. The L7 Proxy Connector will show online when it is ready to be used.