Tetra File-Log Agent Copy File Command

This topic shows you how the Tetra File-Log Agent can be used to download and drop a file to a designated folder by utilizing Tetra Data Platform Command Service.

Prerequisites

To complete these instructions you'll need:

• Get a JWT Token and Org Slug from the Tetra Data Platform
• Set up a GDC or CDC Agent from Tetra Data Platform with queue enabled
• Access to a tool that will allow you to run REST calls, such as curl, Postman, or some other method

📘

NOTE:

FLA Drop File Command is available for File-Log Agent version 4.2.0 and above

Have File-Log Agent up and running

Enable Receive Commands option

Send Drop File Command Payload

You can invoke Tetra Command API to send a command. Below is the sample of the command,
To send a successful command, the following items are required:

1. Tetra Command API: POST https://api.tetrascience.com/v1/commands
2. Headers, including x-or-slug and ts-auth-token
3. Valid Request Body

// POST https://api.tetrascience.com/v1/commands

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Request body:
{
  "targetId": "<agent-id>",
  "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
  "metadata": {},
  "expiresAt": "<command expiration datetimeoffset in ISO-8601 format, e.g. 2022-12-31T23:58:43.749Z>",
  "payload": {
    "source": "https://...",
    "destination": "c:\\tmp\\test.txt"
  }
}

Request Body Data Field Description

Drop File Command Payload JSON Schema

{
    "$schema": "https://json-schema.org/2020-12/schema",
    "$id": "https://tetrascience.com/command.schema.json/v1",
    "title": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "description": "Payload Schema for Command TetraScience.Agent.file-log.DropFile.CopyFile",
    "type": "object",
    "properties": { 
        "source": { "type": "string", "format": "uri", "minLength": 1 },
        "destination": { "type": "string", "minLength": 1 },
        "createPath": { "type": "boolean" },
        "overwrite": { "type": "boolean" },
        "rawMd5Checksum": { "type": "string" }
    },
    "required": ["source" , "destination"]
}

Command Validation

The Tetra Data Platform validates the command JSON schema and the included payload JSON schema
If either of them is invalid, Tetra Data Platform will return the response as below

{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Invalid request payload input"
}

If both schema are valid, Tetra Data Platform returns response containing the new generated command Id

{
    "id": "<new_command_id>",
    "orgSlug": "<org_slug>",
    "targetId": "<agent_id>",
    "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "expiresAt": "2022-12-31T23:58:43.749Z",
    "metadata": {
        "key1": "value1",
        "key2": "value2"
    },
    "status": "PENDING",
    "requestBody": {
      ...
    }
}

When the Agent receives the Dropfile Command, it will start to download the file and validate content of the JSON payload and the source file.

If any of the following rules doesn't meet, the Agent will send a FAILURE status back to Tetra Data Platform.

1. The command is not expired.
2. The file size won't exceed 100 MB
3. The Agent group account has write permission of the destination folder
4. The MD5 checksum in the command should match with the source file's

After the Agent successfully download and drop the file to the destination folder, the Agent will send SUCCESS status back to Tetra Data Platform.

Best Practices

1. Avoid setting the destination to a folder that a File-Log Agent is watching as this will potentially create a loop, wherein the file downloaded by the agent through the CopyFile command will be uploaded by the agent to TDP again
2. Calculate and send the MD5 checksum in the payload for integrity check

Including the CopyFile command in TDP task scripts

The ts-sdk library provides two functions which greatly simplify creating and sending CopyFile commands in TDP Task Scripts.

context.get_presigned_url - This is used to create a pre-signed S3 URL for a TDP File Pointer.

context.run_command - This is used to send the command to the agent, and receive the response.

With the pre-signed URL, you can construct the payload as described above and use the run_command function to send it to the agent.