API DocumentationProduct Documentation

FLA Programmatic Agent Creation and Configuration

Suggest Edits

This topic shows you how the Tetra File-Log Agent can be created and configured using APIs to enable automation.

Prerequisites

To complete these instructions you'll need:

  • A JWT Token and Org Slug from the TDP.
  • The Cloud Data Connector ID
  • Access to a tool that will allow you to run REST calls, such as curl, Postman, or some other method.

📘

NOTE:

Online configuration is available for File-Log Agent 4.1 or later. Note also that a Datahub with the L7 proxy is required for Agents that don’t have Internet access.

Get the Connector ID, Org Slug, and JWT Token

To get the connector ID, Org slug, and JWT Token, complete the following steps.

📘

NOTE:

These instructions explain how to use a CDC to connect the Agent, but you can create and configure Agents in a Datahub GDC in the same way.

  1. To start, you will need an existing Cloud Data Connector for the Agent to connect to.
1549

Cloud Data Connector

  1. Once you've created the Cloud Data Connector in the platform, you can retrieve its ID by clicking "Create a New Agent." Another page appears that shows the connector's ID in the URL as highlighted below. Copy this URL.
685
  1. Before proceeding, you'll need your org slug and a JWT authentication token for the Agents API. Follow the instructions here to get them.

Create an Agent using the API

Now that you have a Connector ID, your org slug, and JWT authentication token, you can use the Agents API to create an Agent .

  1. Use a tool such as curl, Postman, (or your own code) to enter code that is similar to what is below. 
// POST https://api.tetrascience.com/v1/agents

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Request body:
{
  "sourceType": null,
  "name": "File-Log Agent",
  "description": "your agent description",
  "tags": [
    "example-tag-1"
  ],
  "metadata": {
    "meta1": "value1"
  },
  "type": "file-log",
  "integrationId": "<your connector id>",
  "integrationType": "udi",
  "labels": {
    "labelsToAdd": [
      {
        "name": "example-label-1",
        "value": "example label 1"
      }
    ]
  }
}

// Response body:
{
  "id": "<the-agent-id>",
  "name": "File-Log Agent",
  "description": "your agent description",
  "sourceType": null,
  "type": "file-log",
  "integrationType": "udi",
  "integrationId": "<your-connector-id>",
  "datahubId": null,
  "sourceId": "<internal-source-id>",
  "isEnabled": true,
  "queue": null,
  "tags": [
    "example-tag-1"
  ],
  "metadata": {
    "meta1": "value1"
  },
    "config": null,
    "configStatus": null,
    "configStatusUpdatedAt": null,
    "configStatusError": null,
    "createdAt": "2022-02-12T00:50:04.159Z",
    "updatedAt": "2022-02-12T00:50:04.159Z"
	}
}
  1. This will create the Agent in the platform. Note that the response will contain the Agent ID. Copy it because you will need it for the next step.

Install the FLA on the Target Machine and Configure it to Receive Commands

Here is how you install the FLA on your target machine and configure it to receive commands.

  1. Install the File-Log Agent on the target machine.
  2. Open the File-Log Agent Management Console and start to configure it, using the ID from the response above as the Agent Id, the Cloud Data Connector endpoint as the Connection Url, and your Org Slug and Authorization token.
  3. Turn on "Receive Commands", to allow the Agent to receive configuration commands from the platform.
1179

FLA Interface

  1. You will see 'Connected' once the Agent is configured correctly.
  2. When finished, start the Agent service.

Enabling the Command Queue on TDP

Before configuring the Agent programmatically, you will need to enable the command queue on the platform.

  1. Use the following API endpoint to enable the command queue on the TDP for your Agent.
// PUT https://api.tetrascience.com/v1/agents/<your-agent-id>/command-queue

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>

// Request body:
{
  "enabled": true
}

// Response body:
{
  "enabled": true,
  "queue": "<your-queue-url>",
  "visibilityTimeout": 600
}
  1. Now you can use the Agents API to check that your Agent is online (this may take a minute or two from when the Agent service is started). You should also see the Agent's queue information in the response:
// GET https://api.tetrascience.com/v1/agents/<your-agent-id>

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Response body:
{
    "id": "<your-agent-id>",
    "name": "File-Log Agent",
    "description": "some description",
    "sourceType": null,
    "type": "file-log",
    "integrationType": "udi",
    "integrationId": "<your-connector-id>",
    "datahubId": null,
    "sourceId": "<internal-source-id>",
    "isEnabled": true,
    "tags": [
        "example-tag-1"
    ],
    "metadata": {
        "meta1": "value1"
    },
    "config": null,
    "configStatus": null,
    "configStatusUpdatedAt": null,
    "configStatusError": null,
    "configChangedBy": null,
    "configChangedAt": null,
    "createdAt": "2022-05-18T17:36:33.447Z",
    "updatedAt": "2022-05-18T17:36:33.447Z",
    "labels": [
        {
            "id": 13409,
            "name": "example-label-1",
            "value": "example label 1"
        }
    ],
    "status": "Online",
    "version": "v4.1.0",
    "liveType": "file-log",
    "host": {
        "ip": null,
        "name": null
    },
    "queue": {
      "enabled": true,
      "queue": "<your-queue-url>",
      "visibilityTimeout": 600
    }
}

Updating the Agent's Configuration via the API

To update the Agent's configuration via the API, complete the following steps.

🚧

NOTE:

This will remove any paths not included in the JSON. To preserve existing paths, you must include them each time you POST.

  1. POST a new config to the Agent using the API - the expected JSON format follows the code sample below.
// POST https://api.tetrascience.com/v1/agents/{agentId}/configuration

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Request body:
{
  "services_enabled": [
    "fileWatcher"
  ],
  "services_configuration": {
    "fileWatcher": {
      "paths": [
        {
          "path": "c:\\test-path-1",
          "patterns": [
            "*"
          ],
          "file_watch_mode": "file",
          "source_type": "unknown",
          "metadata": {
            "meta1": "value1"
          },
          "tags": "path-tag-1",
          "interval": 30,
          "start_date": "2022-01-01"
        },
        {
          "path": "c:\\test-path-2",
          "patterns": [
            "*"
          ],
          "file_watch_mode": "file",
          "source_type": "unknown",
          "metadata": {
            "meta2": "value2"
          },
          "tags": "path-tag-2",
          "interval": 30,
          "start_date": "2022-01-01"
        }
      ],
      "use_path_configuration": true
    }
  }
}
  
// Response body:
{
  "commandId": "<some-command-id>"
}
  1. If you still have the Agent UI open, you will soon see the updated configuration reflected there. There are also several ways to retrieve the Agent's configuration using the API. This is covered in the next section.

Retrieving Agent Information and Configuration

The GET Agent details API will include its latest applied configuration:

// GET https://api.tetrascience.com/v1/agents/<your-agent-id>

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Response body:
{
  "id": "d9ee8ce3-78f2-494a-82d1-b391c30eb879",
  "name": "File-Log Agent",
  "description": "some description",
  "sourceType": null,
  "type": "file-log",
  "integrationType": "udi",
  "integrationId": "c69bd7b2-58b9-488b-9d23-4d4eb416f785",
  "datahubId": null,
  "sourceId": "a604d02a-ba51-469e-8706-b4efa7d2976a",
  "isEnabled": true,
  "tags": [
    "example-tag-1"
  ],
  "metadata": {
    "meta1": "value1"
  },
  "config": {
    "services_enabled": [
      "fileWatcher"
    ],
    "services_configuration": {
      "fileWatcher": {
        "paths": [
          {
            "path": "c:\\test-path-1",
            "tags": "path-tag-1",
            "interval": 30,
            "metadata": {
              "meta1": "value1"
            },
            "patterns": [
              "*"
            ],
            "start_date": "2022-01-01",
            "source_type": "unknown",
            "file_watch_mode": "file"
          },
          {
            "path": "c:\\test-path-2",
            "tags": "path-tag-2",
            "interval": 30,
            "metadata": {
              "meta2": "value2"
            },
            "patterns": [
              "*"
            ],
            "start_date": "2022-01-01",
            "source_type": "unknown",
            "file_watch_mode": "file"
          }
        ],
        "interval": 30,
        "start_date": "2022-05-16",
        "use_path_configuration": true
      }
    }
  },
  "configStatus": "SUCCESS",
  "configStatusUpdatedAt": "2022-05-18T17:53:07.386Z",
  "configStatusError": null,
  "configChangedBy": "cloud",
  "configChangedAt": "2022-05-18T17:53:05.033Z",
  "createdAt": "2022-05-18T17:36:33.447Z",
  "updatedAt": "2022-05-18T17:53:04.961Z",
  "labels": [
    {
      "id": 13409,
      "name": "example-label-1",
      "value": "example label 1"
    }
  ],
  "status": "Online",
  "version": "v4.1.0",
  "liveType": "file-log",
  "host": {
    "ip": null,
    "name": null
  },
  "queue": {
    "enabled": true,
    "name": "https://sqs.us-east-2.amazonaws.com/706717599419/onprem-d9ee8ce3-78f2-494a-82d1-b391c30eb879.fifo",
    "visibilityTimeout": 600
  }
}

You can also get just the latest configuration applied to the Agent using the API shown below:

// GET https://api.tetrascience.com/v1/agents/{id}/configuration

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Response body:
{
  "id": "4c483892-40b6-45b9-a352-cc7a46d1cb17",
  "config": {
    "services_enabled": [
      "fileWatcher"
    ],
    "services_configuration": {
      "fileWatcher": {
        "paths": [
          {
            "path": "c:\\online1",
            "tags": "path-tag-1",
            "interval": 30,
            "metadata": {
              "meta1": "value1"
            },
            "patterns": [
              "*.*"
            ],
            "start_date": "2022-01-01",
            "source_type": "something",
            "file_watch_mode": "file"
          }
        ],
        "interval": 30,
        "start_date": "2022-07-07",
        "use_path_configuration": true
      }
    }
  },
  "by": "cloud",
  "at": "2022-07-08T19:04:34.993Z"
}

You can also check the latest cloud configuration, whether it's been applied by the Agent successfully or not (in this example, it was not due to an invalid file_watch_mode):

// GET https://api.tetrascience.com/v1/agents/{id}/configuration?type=cloud

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
{
  "id": "cf9d2576-b073-4744-9101-def5a37b32cf",
  "config": {
    "services_enabled": [
      "fileWatcher"
    ],
    "services_configuration": {
      "fileWatcher": {
        "paths": [
        {
          "path": "c:\\online1",
          "tags": "path-tag-1",
          "interval": 30,
          "metadata": {
            "meta1": "value1"
          },
          "patterns": [
            "*.*"
          ],
          "start_date": "2022-01-01",
          "source_type": "something",
          "file_watch_mode": "INVALID"
        }
      ],
      "use_path_configuration": true
     }
   }
 },
 "by": "cloud",
 "at": "2022-07-08T19:15:43.246Z",
 "commandStatus": "FAILURE",
 "commandId": "cf9d2576-b073-4744-9101-def5a37b32cf"
}

You can also check the Agent's configuration history using the API shown below. Responses are listed from most to least recent. In this example, the earlier configuration command failed because of invalid paths, and the later one succeeded:

// GET https://api.tetrascience.com/v1/agents/{id}/configuration/list

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Response body:
{
  "page": 1,
  "size": 10,
  "hasNext": false,
  "hits": [
    {
      "id": "<command-id>",
      "config": {
        "services_enabled": [
          "fileWatcher"
        ],
        "services_configuration": {
          "fileWatcher": {
            "paths": [
              {
                "path": "c:\\test-path-1",
                "tags": "path-tag-1",
                "interval": 30,
                "metadata": {
                  "meta1": "value1"
                },
                "patterns": [
                  "*"
                ],
                "start_date": "2022-01-01",
                "source_type": "unknown",
                "file_watch_mode": "file"
              },
              {
                "path": "c:\\test-path-2",
                "tags": "path-tag-2",
                "interval": 30,
                "metadata": {
                  "meta2": "value2"
                },
                "patterns": [
                  "*"
                ],
                "start_date": "2022-01-01",
                "source_type": "unknown",
                "file_watch_mode": "file"
              }
            ],
            "interval": 30,
            "start_date": "2022-05-16",
            "use_path_configuration": true
          }
        }
      },
      "by": "cloud",
      "at": "2022-05-18T17:53:05.033Z",
      "commandId": "8819af94-b9ac-4ea9-9540-97328ffae96d",
      "commandStatus": "SUCCESS"
    },
    {
      "id": "f8c11ec3-fbfb-45e5-a358-b23de23714df",
      "config": {
        "services_enabled": [
          "fileWatcher"
        ],
        "services_configuration": {
          "fileWatcher": {
            "paths": [
              {
                "path": "c:\\<your-test-path-1>",
                "tags": "path-tag-1",
                "interval": 30,
                "metadata": {
                  "meta1": "value1"
                },
                "patterns": [
                  "*"
                ],
                "start_date": "2022-01-01",
                "source_type": "unknown",
                "file_watch_mode": "file"
              },
              {
                "path": "c:\\<your-test-path-2>",
                "tags": "path-tag-2",
                "interval": 30,
                "metadata": {
                  "meta2": "value2"
                },
                "patterns": [
                  "*"
                ],
                "start_date": "2022-01-01",
                "source_type": "unknown",
                "file_watch_mode": "file"
              }
            ],
            "use_path_configuration": true
          }
        }
      },
      "by": "cloud",
      "at": "2022-05-18T17:52:27.487Z",
      "commandId": "f8c11ec3-fbfb-45e5-a358-b23de23714df",
      "commandStatus": "FAILURE"
    }
  ]
}

Commands Pending, Failure and Expiration

When a configuration command is waiting to be received and applied by an Agent, the Agents UI indicates this:

438

If a configuration command fails to apply on the Agent - for example, because it had an invalid configuration - this is indicated in the Agents UI as shown:

487

If the Agent does not receive a command within 4 minutes, this is also shown in the Agents UI:

492

Finally, when the configuration is applied successfully, this will be shown:

438

Cloud Configuration superseded by local

Because the Agent can be configured locally as well as by these commands, it may receive a cloud command that's already been superseded by a later local configuration. When that happens, the Agent will ignore the superseded configuration, and it will be considered Successful.

Configuration via TDP UI

The APIs listed above are the preferred way to configure your Agents, because they support repeatability and automation. You can also configure Agents via the TDP UI:

  1. Find your Agent on the Agents page in TDP
  2. From the right of the Tetra File-Log Agent's entry, select the More option.
  3. Select Configure.
  4. From the Configure page, set the Path Configurations and File Watcher settings. For more details, see the Tetra File-Log Agent Installation Guide.
  5. Click Apply.
624 221 1153

Updated over 1 year ago