Tetra File-Log Agent Copy File Command

The following procedure shows how to download and drop a file to a designated folder programmatically by using the Tetra File-Log Agent and the Tetra Data Platform (TDP) Command Service.

Prerequisites

To complete the procedure, you must first do the following:

• Get a JWT Token and Org Slug from the Tetra Data Platform
• Create and configure a Tetra File-Log Agent that has its queue enabled
• Access to a tool that will allow you to run REST calls, such as curl, Postman, or some other method

📘

NOTE:

FLA Drop File Command is available for File-Log Agent version 4.2.0 and above

Have a Tetra File-Log Agent Up and Running

Enable the Receive Commands option. For instructions, see Receive Commands/Enable Queue in the Tetra File-Log Agent Installation Guide.

Send Drop File Command Payload

Invoke the Command Service API to send a command. To send a successful command, the following items are required:

1. Tetra Command API: POST https://api.tetrascience.com/v1/commands
2. Headers, including x-or-slug and ts-auth-token
3. Valid Request Body

Drop File Command Example

// POST https://api.tetrascience.com/v1/commands

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Request body:
{
  "targetId": "<agent-id>",
  "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
  "metadata": {},
  "expiresAt": "<command expiration datetimeoffset in ISO-8601 format, e.g. 2022-12-31T23:58:43.749Z>",
  "payload": {
    "source": "https://...",
    "destination": "c:\\tmp\\test.txt"
  }
}

Request Body Data Field Description

Drop File Command Payload JSON Schema

{
    "$schema": "https://json-schema.org/2020-12/schema",
    "$id": "https://tetrascience.com/command.schema.json/v1",
    "title": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "description": "Payload Schema for Command TetraScience.Agent.file-log.DropFile.CopyFile",
    "type": "object",
    "properties": { 
        "source": { "type": "string", "format": "uri", "minLength": 1 },
        "destination": { "type": "string", "minLength": 1 },
        "createPath": { "type": "boolean" },
        "overwrite": { "type": "boolean" },
        "rawMd5Checksum": { "type": "string" }
    },
    "required": ["source" , "destination"]
}

Command Validation

The TDP validates the command JSON schema and the included payload JSON schema. If either of them aren't valid, the TDP returns the following response:

{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Invalid request payload input"
}

If both schemas are valid, the TDP returns a response that contains the new generated command Id, such as the following:

{
    "id": "<new_command_id>",
    "orgSlug": "<org_slug>",
    "targetId": "<agent_id>",
    "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "expiresAt": "2022-12-31T23:58:43.749Z",
    "metadata": {
        "key1": "value1",
        "key2": "value2"
    },
    "status": "PENDING",
    "requestBody": {
      ...
    }
}

When the Agent receives the Drop File Command, it will start to download the file and validate content of the JSON payload and the source file.

If any of the following rules aren't met, the Agent sends a FAILURE status back to the TDP:

1. The command is not expired.
2. The file size won't exceed 100 MB
3. The Agent group account has write permission of the destination folder
4. The MD5 checksum in the command should match with the source file's

After the Agent successfully downloads and drops the file to the destination folder, the Agent then sends a SUCCESS status back to TDP.

Best Practices

1. Avoid setting the destination to a folder that a File-Log Agent is watching. Doing this can create a loop where the file downloaded by the Agent through the CopyFile command will be uploaded by the Agent to the TDP again.
2. Calculate and send the MD5 checksum in the payload for integrity check

Including the CopyFile command in TDP task scripts

The ts-sdk library provides two functions that greatly simplify creating and sending Copy File commands in TDP task scripts:

• context.get_presigned_url - This is used to create a pre-signed S3 URL for a TDP File Pointer.

• context.run_command - This is used to send the command to the agent, and receive the response.

With the pre-signed URL, you can construct the payload as described above and use the run_command function to send it to the Agent.