After the Tetra Agent extracts copies of the data files from the data source, it sends those files to the Tetra Data Platform. This topic provides details on how to configure integrations and connectors to transfer the data.

Tetra Agent Integration and Connector Overview

TetraScience provides several different ways for Tetra Agents to integrate with the Tetra Data Platform.

Ways that Tetra Agents Can Integrate with TDP

Selecting a Connector: GDC or TDP

To determine which type of connector to use, you’ll need to know two things:

• Is the Tetra Agent installed on a machine that has internet access?
• Are the files to be transferred to the data lake large (more than 500 MB) or small (less than 500 MB)?

Connector Decision Table

• If the agent has internet access, use "No Connector", and make sure that you have the S3 Direct Upload configuration enabled.
• If the agent has internet access, and files are small, you can use "No Connector" without S3 Direct Upload.
• If the agent does not have internet access, and the files are large, use the GDC, which should be on the Data Hub. You will also need to use the L7 Proxy Connector. The L7 Proxy Connector (L7C) is a simple proxy server that uploads files directly to the data lake. L7C accommodates large files. The L7 Proxy Connector should also be set up on the Data Hub. Make sure that you have the S3 Direct Upload configuration enabled.
• If the agent does not have internet access, you cannot configure an L7, and the files are small, you can use the Generic Data Connector (GDC) on the DataHub without S3 Direct Upload.

📘

NOTE

Typically, the S3 Direct Upload is preferred -- even in cases where it’s not strictly required by file size -- because of its performance.

• The Agent's host machine will need to be on Windows 10 or newer, or Windows Server 2016 for the Agent to connect to it properly.

Connector URL

After determining which connector to use you will need to specify the Connection URL. Examples of the URL from GDC or TDP as below

• GDC URL is http://10.100.1.1:8888/generic-connector/v1/agent
• TDP URL is https://api.myorg-dev.com/v1/data-acquisition/agents

Connector/Integration Configuration Instructions

Generic Data Connector (GDC)

Tetra Agents can be integrated with the Tetra Data Platform through Tetra Data Hub. The Tetra Data Hub is an on-premise server that facilitates securely transferring data extracted from a Tetra Agent to the Tetra Data Platform. The Agent connects to the Data Hub through a data connector.

Prerequisites

• A Tetra Data Hub must be set up.
• A connector must be added to the Tetra Data Hub.
• The Data Hub and Data Connector must be online.

Procedures

Because the Tetra Agent transfers data with a Tetra Data Hub Connector, you must register the Tetra Agent first on the Tetra Data Platform with a specific connector.

Here's how to add a Tetra Agent to your existing connector on the Tetra Data Platform:

1. Use your web browser to log into the Tetra Data Platform.
2. On the Hamburger menu, select Data Sources > Data Hub.
3. In the Data Management page, find the connector you want to add the Tetra Agent to. Click on the "menu" icon (1), then click "Edit" (2).
4. Scroll to the Agent section, then click Add Agent.
5. The Agent ID is auto-generated by the Tetra Data Platform. You just need to give the Agent a name. Optionally, you can also specify Agent metadata and tags for quick identification in the Data Management page.
6. Click Save.
7. The new Agent appears with the connector on the Data Management page. The status of the Agent changes to online when the Agent service starts and successfully connects to the connector.

Using the "No Connector" Option (TDP)

Besides the connection through the Data Hub/Data Connector, the Agent can directly upload the files to the Tetra Data Platform.

Prerequisites

• Client Organization Slug (orgSlug) is created on Tetra Data Platform
• A user account is created in that Organization Slug

Procedures

1. Log in to the Tetra Data Platform.
2. Create an Agent entry in the TDP:
   a. Select Data Sources > Agents from the TDP Hamburger menu.
   b. Click “Create New Agent” in the top right corner of the screen.
   c. In the Add Agent popup, leave the option as “No Connector”, then click Next.
   d. Enter the name, description, and Agent type. If you need to add an attribute (such as a label), click the Add/Edit Attributes button.
   e. Click Create Agent.
   f. The Agent is created in TDP. Click Install Agent Now
   g. Copy the Agent ID and Connection Url. In your agent, enter those values in Agent Id and Connection Url, respectively, and add your organization in the Org Slug.

To create an Authorization token for a Service User, see this link.

Copy this authorization token into the Authorization field on your agent. You should now see a Connected message next to your URL:

3. Provide Integration Name and click Create

4. Provide detailed information.

Agent Id is generated as default.

a. Select Source Type. Source Type is mandatory. The default is unknown.
b. Provide Name and Description.
c, Provide Meta Data and Tags, Provide metadata and tags if it is needed and click Finish to complete.
c. The Agent is created. The Agent Id will be used in the Agent configuration.

5. Now you have both Agent Id and JSON Token. It is ready to set up the Agent to upload the files to the Tetra Data Platform.

L7 Proxy Connector

It is optional to create an L7 Proxy Connector along with Generic Data Connector. The purpose of L7 Proxy Connector to allow end-user to upload large files (size is larger than 500 MB) to TetraScience Data Platform using the Generic Data Connector option

❗️

For large files (more than 500MB)

Remember to select S3 direct upload in the agent and point to the L7 Proxy connector. L7 Proxy without S3 direct upload will fail at uploading large files.

The end-user can follow the instruction below to create an L7 Proxy Connector.

1. Go to TetraScience platform page on your browser
2. Click on the upper left corner to bring up the sidebar and click "Data Hub"
3. Click Add Connector
4. Select L7 Proxy Connector from the dropdown
5. Fill in the Name, and provide the Port number. The default is 3128. Please double check with System Administrator to make sure this port is available
6. Click Save, then wait a couple of minutes. The L7 Proxy Connector will show online. Now, it is ready.