Tetra File-Log Agent Frequently Asked Questions

This topic contains Frequently Asked Questions (FAQs) for the Tetra File-Log Agent (FLA).

What is the File-Log Group User and how it is used by the Agent?

One of the core applications in File-Log Agent runs as a Windows Service. This application runs in the background to detect the changes in files and folders. When configuring the File-Log Agent, the user needs to specify which account (File-Log Group User) runs that Windows Services application.

The File-Log Group User account could be a local Windows User Account, a Service Account, or a Domain User Account.

Please check the Microsoft guideline for selecting the account.

Also, the File-Log Group User account should have Log on as service right and the account has read permission as the minimum for the path under monitoring.

If the File-Log Group User account is left empty, the Agent will use the local system account as default.

539539

What is the required permissions for folders or files?

The Tetra File-Log Agent requires that the File-Log Group User has at least read permission on the files and folders, including the subfolders and files contained in those folders.

If the File-Log Group User doesn't have read permission on the paths defined in the Management Console, the Agent won't start until the permission issue is fixed.

If the File-Log Group User doesn't have read permission on the subfolders or the files contained in the paths defined in the Management Console, the Agent will skip those folders and files. The error message is written to the Agent log if the path is local. If the path is UNC, the Agent won't be able to generate any error message due to limited permission.

What should the user do if there is a connection issue when starting the Agent?

When the user starts the Agent, the Agent will verify the connection to the TetraScience platform. If the user sees the following message, it means the Agent is not able to connect to the TDP.

379379

The user can take the following steps to investigate and correct that issue.

  • Verify the configuration on the TDP.
  • Verify the Generic Data Connector or User-defined Integration is accessible from the computer hosting the File-Log Agent.
  • Verify the Agent ID and URL (including the protocol, IP and Port) are correct.
  • Verify the protocol used for Data Connection.
    If using HTTP, the Agent can take the connection URL as HTTP://IP/port or just IP/Port
    If using HTTPS, the user has to provide HTTPS explicitly, otherwise, the Agent will assume the protocol is HTTP.

Why are files not uploaded?

There could be multiple causes for this.

  • Understand how the Agent detects the file or folder changes. Please check the detecting logic from fileWatcherService and logWatcherService respectively. (The LogWatcher Service is only available for versions before the File-Log Agent 4.1.0 release.)
  • Check the defined patterns. Please check the common usage of Glob pattern implemented in File-Log Agent.
  • Make sure Start Date is correct.
  • Make sure Group User Temp folder has enough free space to store the largest file being uploaded by the File-Log Agent.

How can I change TEMP folder?

The Group User Temp folder is used to move the file into by the File-Log Agent before being uploaded to TDP.
Theoretically the Temp folder should have enough free space to store the largest file being uploaded. For this purpose if the Temp folder location/path needs to be changed, please refer to these articles:

How do I specify the Data Connection Protocol?

The user can select HTTP or HTTPS to connect to TDP when using the Generic Data Connector. It is part of the configuration setting in the Data Connector.

781781

The user can set the port for HTTP or HTTPS, as long as those ports are available.

When connecting to TDP through User-Defined Integration, only HTTPS is supported.

How do I upgrade from the previous version of the File-Log Agent?

❗️

Stop the existing Agent Windows service

Make sure you stop the previous agent before upgrade/reinstall.

The user can choose Upgrade when installing the File-Log Agent. The Agent will retain the configuration setting and Sqlite database from the previous version.

623623

If the user selects New Installation during installation. The File-Log Agent will be installed from scratch. The configuration setting and Sqlite database which stores the file processing history will be refreshed.

What happens if the Source Type, Metadata, or Tag is changed?

❗️

Functionality changes

The Agent has changed this behavior after v3.4.0.
The behavior below is applied to the following versions

  • v3.0.0
  • v3.1.0
  • v3.2.0
  • v3.2.1
  • v3.3.1

From v3.4.0, the behavior that Agent will re-upload existing files when any of Source Type, Metadata or Tags change has been reverted. Instead, there are two changes applied

  1. When the Metadata, Source Type, or Tag is updated, the change will be applied to only the new files or the updated files after the update.

  2. The Agent provides a Re-upload button for users to determine what files should be re-upload explicitly.

The detail is described in v3.x.x installation guide

When the user updates any of the Source Type, Metadata, or Tag, the Agent will reprocess the files.
Taking an example that the Agent is configured using FileWatcher Service. Six files are being processed and uploaded to TetraScience Data Platform.

383383

The user can stop the Agent and add a new Metadata entry. The Agent notifies the user that the files will be reprocessed.

624624

After the files are reprocessed, the Summary page updates accordingly.

376376

Please beware that there have no duplicated files that exist in the Tetra Data Platform. The files have a new version. Taking the example above, there still have six files that exist in TetraScience Data Platform. Each file has two versions.

What should be done if we want to re-upload files to the TDP?

Add new Metadata or tag to trigger the Agent to re-process/re-upload the files.

How can I use the File Path Start Date and File Change Interval Field in the File Watcher Service?

When a file path is under monitoring by File Watcher Service, it needs to determine the following:

  • What are the files in the file path that should be monitored

Besides the glob pattern defined in the file path, another criterion is Start Date (file age). The File-Log Agent monitors the files with their last write time is greater than the date specified in Management Console. It impacts the initial file upload only. The Agent can exclude historical files that the user doesn't want to upload.

  • When the files should be uploaded

The Agent scans the file paths using its predefined time interval. When the Agent detected the file change, the Agent doesn't upload it immediately which tries to avoid uploading any file in an intermediate transition stage. Instead, the Agent has a File Change Interval feature. The Agent compares the File Last Write Time from two continuous scans. If the values are the same, the Agent thinks the file change is done which is eligible for upload. When the file is uploaded, it won't be reloaded unless it is updated.

This setting can be configurated either on the global level which is applied to all of the file paths under monitoring or on the individual file path level. There has an option button on the Windows Management Console for the user to choose.

11101110

How to enable long path for Windows?

In editions of Windows before Windows 10 version 1607, the maximum length for a path is MAX_PATH, which is defined as 260 characters. In the later versions of Windows, changing a registry or using the Group Policy is required to remove the limit.

The following editions of Windows OS support long path:

  • Windows 10 (version 1607 and later)
  • Windows Server 2016
  • Windows Server 2019

Windows 8.1 and Windows Server 2012 (which is based on Windows 8.1), does not support paths longer than 260 characters.

Please read the following posts regarding how to remove the 260-character limit

How do I report errors to TetraScience?

If something unexpected occurs, contact us. Provide the following two items.

  • Complete Log folder, which is located in the File-Log Agent installation folder. We suggest that you compress the entire folder.
503503
  • Sqlite Database file, applicationdb.sqlite, which is located in the Database folder under Bin.
503503

Which AWS Endpoints Does the File-Log Agent Use?

The Tetra File-Log Agent uses these AWS endpoints:

AWS EndpointDescriptionWhen Required
s3.[region].amazonaws.comUploads filesWhen the Enable S3 Direct Upload option is selected
sqs.[region].amazonaws.comFetches the command message and then returns the command processing statusWhen the Enable Queue option is selected
logs.[region].amazonaws.comPosts Agent Heart Beats and Agent logsWhen the Enable S3 Direct Upload option is selected
monitoring.[region].amazonaws.comSends Metrics Data (such as CPU, Memory, and Disk usage)When the Enable S3 Direct Upload option is selected

What are the acceptable alphabetic characters in the paths and files watched by FLA?

Alphabetic characters A-Z and a-z are supported in the paths and file names watched. If the path or filename has any other alphabets, accents, dieresis etc. the FLA may not behave correctly and it is suggested not to use these alphabetic characters in paths/files which FLA is watching.