The following procedure shows how to download and drop a file to a designated folder programmatically by using the Tetra File-Log Agent and the Tetra Data Platform (TDP) Command Service.

📘

NOTE

The Copy File Command is available for Tetra File-Log Agent versions 4.2.0 and higher only.

Prerequisites

The Tetra File-Log Agent Copy File Command requires the following:

• A running Tetra File-Log Agent v4.2.0 or higher installed and configured with its queue enabled.
• A valid JSON Web Token (JWT) for the service user running the command. To create a JWT, see Generate a JWT for a Service User.
• Your TDP organization's slug (ORG SLUG). To find your org slug, see View Organization Details.
• Access to a tool that will allow you to run REST API calls, such as curl or Postman.

Step 1: Verify That the Agent Can Receive Commands

To allow the Agent to securely receive and run commands from the TDP, do the following:

1. From the Tetra File-Log Agent Management Console, in the left navigation menu, under Menu, choose Configuration.
2. Verify that the Receive Commands toggle is set to Yes (the default value is set to No).

For more information, see Set Up the Agent Configuration in the Tetra File-Log Agent Installation Guide.

Step 2: Run the Copy File Command

To run the Tetra File-Log Agent Copy File Command, do the following:

1. Create a valid Create Command in your code editor by using the following Copy File Command Example code snippet.

📘

NOTE

Keep in mind the following when creating your commands:

• Don't set the destination to a folder that a Tetra File-Log Agent is watching and scanning. Doing this can create a loop, where the file downloaded by the Agent through the Copy File Command is then also uploaded by the Agent to the TDP again.
• It's a best practice to always calculate and send the file's MD5 checksum in the payload. Including the MD5 checksum value helps ensure data integrity.

2. Copy and run the updated command by using a tool that will allow you to run REST API calls, such as curl or Postman.

When the Agent receives the Copy File Command, it will start to download the file and validate the content of the JSON payload and the source file. After the Agent successfully completes the copy file action, it then sends a SUCCESS status back to the TDP.

For more information, see Command Validation.

Copy File Command Example

🚧

IMPORTANT

You must enter values for the following variables:

• ForPOST, enter https://api.tetrascience.com/v1/commands
• For x-org-slug, enter your organization's slug.
• For ts-auth-token, enter your service user's JWT.
• Make sure that you enter a valid Request body that includes all of the Required fields listed in the Request Body Data Fields table.

// POST https://api.tetrascience.com/v1/commands

// Headers:
x-org-slug: <your-org-slug>
ts-auth-token: <your-auth-token>
  
// Request body:
{
  "targetId": "<agent-id>",
  "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
  "metadata": {},
  "expiresAt": "<command expiration datetimeoffset in ISO-8601 format, e.g. 2022-12-31T23:58:43.749Z>",
  "payload": {
    "source": "https://...",
    "destination": "c:\\tmp\\test.txt"
  }
}

Request Body Data Fields

Drop File Command Payload JSON Schema

{
    "$schema": "https://json-schema.org/2020-12/schema",
    "$id": "https://tetrascience.com/command.schema.json/v1",
    "title": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "description": "Payload Schema for Command TetraScience.Agent.file-log.DropFile.CopyFile",
    "type": "object",
    "properties": { 
        "source": { "type": "string", "format": "uri", "minLength": 1 },
        "destination": { "type": "string", "minLength": 1 },
        "createPath": { "type": "boolean" },
        "overwrite": { "type": "boolean" },
        "rawMd5Checksum": { "type": "string" }
    },
    "required": ["source" , "destination"]
}

Command Validation

The TDP validates the command JSON schema and the included payload JSON schema. The following are examples of successful and failed Tetra File-Log Agent Copy File Command request responses.

Successful Request Responses

If both schemas are valid, the TDP returns a response that contains the new generated command Id, such as the following:

{
    "id": "<new_command_id>",
    "orgSlug": "<org_slug>",
    "targetId": "<agent_id>",
    "action": "TetraScience.Agent.file-log.DropFile.CopyFile",
    "expiresAt": "2022-12-31T23:58:43.749Z",
    "metadata": {
        "key1": "value1",
        "key2": "value2"
    },
    "status": "PENDING",
    "requestBody": {
      ...
    }
}

Failed Request Responses

If either of the schemas aren't valid, the TDP returns the following response:

{
    "statusCode": 400,
    "error": "Bad Request",
    "message": "Invalid request payload input"
}

To resolve common 400 errors, see the Troubleshooting section.

Including the Copy File Command in Task Scripts

To have the Copy File Command interact with task scripts you will need to use the context object. The context object provides functions for you to read files, write files, update metadata, get pipeline configuration information, and more.

The ts-sdk library provides two Context API functions that greatly simplify creating and sending Copy File commands in task scripts:

• context.get_presigned_url: Returns a time-limited HTTPS URL that can be used to access a file.

• context.run_command: Invokes the remote command on the Agent and returns its response.

With the pre-signed URL, you can construct the payload as described in the Copy File Command Example and use the run_command function to send it to the Agent.

Troubleshooting

The Tetra File-Log Agent will send a FAILURE status back to the TDP after it receives a Copy File Command under any of the following conditions: