This topic contains Frequently Asked Questions (FAQs) for the Tetra File-Log Agent (FLA).

What is the File-Log Group User and how it is used by the Agent?

One of the core applications in File-Log Agent runs as a Windows Service. This application runs in the background to detect the changes in files and folders. When configuring the File-Log Agent, the user needs to specify which account (File-Log Group User) runs that Windows Services application.

The File-Log Group User account could be a local Windows User Account, a Service Account, or a Domain User Account.

Please check the Microsoft guideline for selecting the account.

Also, the File-Log Group User account should have Log on as service right and the account has read permission as the minimum for the path under monitoring.

If the File-Log Group User account is left empty, the Agent will use the local system account as default.

What is the required permissions for folders or files?

The Tetra File-Log Agent requires that the File-Log Group User has at least read permission on the files and folders, including the subfolders and files contained in those folders.

If the File-Log Group User doesn't have read permission on the paths defined in the Management Console, the Agent won't start until the permission issue is fixed.

If the File-Log Group User doesn't have read permission on the subfolders or the files contained in the paths defined in the Management Console, the Agent will skip those folders and files. The error message is written to the Agent log if the path is local. If the path is UNC, the Agent won't be able to generate any error message due to limited permission.

What should the user do if there is a connection issue when starting the Agent?

When the user starts the Agent, the Agent will verify the connection to the TetraScience platform. If the user sees the following message, it means the Agent is not able to connect to the TDP.

The user can take the following steps to investigate and correct that issue.

• Verify the configuration on the TDP.
• Verify the Generic Data Connector or User-defined Integration is accessible from the computer hosting the File-Log Agent.
• Verify the Agent ID and URL (including the protocol, IP and Port) are correct.
• Verify the protocol used for Data Connection.
  If using HTTP, the Agent can take the connection URL as HTTP://IP/port or just IP/Port
  If using HTTPS, the user has to provide HTTPS explicitly, otherwise, the Agent will assume the protocol is HTTP.

Why are files not uploading?

There could be multiple causes for this.

• Understand how the Agent detects the file or folder changes. Please check the detecting logic from fileWatcherService and logWatcherService respectively. (The LogWatcher Service is only available for versions before the File-Log Agent 4.1.0 release.)
• Check the defined patterns. Please check the common usage of Glob pattern implemented in File-Log Agent.
• Make sure Start Date is correct.
• Make sure Group User Temp folder has enough free space to store the largest file being uploaded by the File-Log Agent.

If there is a file upload error, will the agent try to upload the file again? And if so, how retries will there be?

If a file upload error occurs, the Tetra File-Log Agent automatically retries uploading the file to the TDP. The Agent retries uploading the file until the upload is successful or the file can't be found. If the Agent can't find a file, that file's status shows as Inactive.

How can I change the TEMP folder?

The Tetra File-Log Agent uses the Group User TEMP folder to temporarily store files before uploading them to the TDP. If you need to change the location or path of the TEMP folder, you can set the environment variables for the Windows user that the Agent will run as. It's recommended to create a service user and set the TMP and TEMP environment variables for that user.

If you have Administrator privileges on the Agent's host server, you can do this by running the following commands in the local terminal:

• runas /user:<username>.<domain> "cmd /c setx TEMP z:\temp"

• runas /user:<username>.<domain> "cmd /c setx TMP z:\temp"

For more information, see How to change TEMP folder and allocate free space on your system disk and Change location of TEMP files folder to another drive in the Microsoft documentation.

📘

NOTE

Make sure that the TEMP folder has enough free space to accommodate the largest files being uploaded.

How do I specify the Data Connection Protocol?

The user can select HTTP or HTTPS to connect to TDP when using the Generic Data Connector. It is part of the configuration setting in the Data Connector.

The user can set the port for HTTP or HTTPS, as long as those ports are available.

When connecting to TDP through User-Defined Integration, only HTTPS is supported.

How do I upgrade from the previous version of the File-Log Agent?

❗️

Stop the existing Agent Windows service

Make sure you stop the previous agent before upgrade/reinstall.

The user can choose Upgrade when installing the File-Log Agent. The Agent will retain the configuration setting and Sqlite database from the previous version.

If the user selects New Installation during installation. The File-Log Agent will be installed from scratch. The configuration setting and Sqlite database which stores the file processing history will be refreshed.

What happens if the Source Type, Metadata, or Tag is changed?

❗️

Functionality changes

The Agent has changed this behavior after v3.4.0.
The behavior below is applied to the following versions

• v3.0.0
• v3.1.0
• v3.2.0
• v3.2.1
• v3.3.1

From v3.4.0, the behavior that Agent will re-upload existing files when any of Source Type, Metadata or Tags change has been reverted. Instead, there are two changes applied

1. When the Metadata, Source Type, or Tag is updated, the change will be applied to only the new files or the updated files after the update.

2. The Agent provides a Re-upload button for users to determine what files should be re-upload explicitly.

The detail is described in v3.x.x installation guide

When the user updates any of the Source Type, Metadata, or Tag, the Agent will reprocess the files.
Taking an example that the Agent is configured using FileWatcher Service. Six files are being processed and uploaded to TetraScience Data Platform.

The user can stop the Agent and add a new Metadata entry. The Agent notifies the user that the files will be reprocessed.

After the files are reprocessed, the Summary page updates accordingly.

Please beware that there have no duplicated files that exist in the Tetra Data Platform. The files have a new version. Taking the example above, there still have six files that exist in TetraScience Data Platform. Each file has two versions.

What should be done if we want to re-upload files to the TDP?

Add new Metadata or tag to trigger the Agent to re-process/re-upload the files.

How can I use the File Path Start Date and File Change Interval Field in the File Watcher Service?

When a file path is under monitoring by File Watcher Service, it needs to determine the following:

• What are the files in the file path that should be monitored

Besides the glob pattern defined in the file path, another criterion is Start Date (file age). The File-Log Agent monitors the files with their last write time is greater than the date specified in Management Console. It impacts the initial file upload only. The Agent can exclude historical files that the user doesn't want to upload.

• When the files should be uploaded

The Agent scans the file paths using its predefined time interval. When the Agent detected the file change, the Agent doesn't upload it immediately which tries to avoid uploading any file in an intermediate transition stage. Instead, the Agent has a File Change Interval feature. The Agent compares the File Last Write Time from two continuous scans. If the values are the same, the Agent thinks the file change is done which is eligible for upload. When the file is uploaded, it won't be reloaded unless it is updated.

This setting can be configurated either on the global level which is applied to all of the file paths under monitoring or on the individual file path level. There has an option button on the Windows Management Console for the user to choose.

📘

NOTE

Global path-level Start Date and Interval (Second) settings aren’t available in FLA management console v4.1.0 and later. These two global path-level settings are scheduled to be deprecated in TDP v3.6.0. For more information, see File-Log Agent Global Path-Level Start Date and Interval (Seconds) Settings Deprecation.

How do I enable long path for Windows?

In editions of Windows before Windows 10 version 1607, the maximum length for a path is MAX_PATH, which is defined as 260 characters. In the later versions of Windows, changing a registry or using the Group Policy is required to remove the limit.

The following editions of Windows OS support long path:

• Windows 10 (version 1607 and later)
• Windows Server 2016
• Windows Server 2019

Windows 8.1 and Windows Server 2012 (which is based on Windows 8.1), does not support paths longer than 260 characters.

Please read the following posts regarding how to remove the 260-character limit

• How to Make Windows 10 Accept File Paths Over 260 Characters
• Enable NTFS long paths in Windows Server 2016 and 2019 by Group Policy

How do I report errors to TetraScience?

If something unexpected occurs, contact us. Provide the following two items.

• Complete Log folder, which is located in the File-Log Agent installation folder. We suggest that you compress the entire folder.

• Sqlite Database file, applicationdb.sqlite, which is located in the Database folder under Bin.

Which AWS Endpoints Does the File-Log Agent Use?

The Tetra File-Log Agent uses these AWS endpoints:

What are the acceptable alphabetic characters in the paths and files watched by FLA?

Alphabetic characters A-Z and a-z are supported in the paths and file names watched. If the path or filename has any other alphabets, accents, dieresis etc. the FLA may not behave correctly and it is suggested not to use these alphabetic characters in paths/files which FLA is watching.

Considerations for enabling archive/delete in File mode

While enabling the archive/delete settings, the following should be considered. These include additional permissions the account running agent will require, edge cases to keep in mind about how to configure archive paths and what happens when a file with same name is archive and added to source folder again:

• To use the archive/delete feature, the account used to run the File-Log agent will need to have the following permissions:
  • “Delete” permission for the path(s) it is monitoring and the archive setting is enabled
  • “Read” and “write” permissions for the archived path that is configured
• When a file is archived, it will create the complete folder structure of the source path of the file inside the archive folder. E.g. if the source file is c:\instrument_1\result_1.csv and the archive folder is c:\archive, where the file is archived, it will be moved to c:\archive<agent ID>\c\instrument_1\result_1.csv
• The “archived path” should not be a path or sub-directory of a path that the File-Log agent (or another File-Log agent) is actively monitoring. Doing this may result in the archived file getting uploaded to TDP again from the archived path.
• In paths where files with the same name can be generated more than once over time, the archive setting will remove the original file from the source path and move it to the archive path after the file has been uploaded and has not changed for the Archive after duration that was configured. After this time period, if a file with the same name that has been uploaded and archived is added to the path again, the Agent will upload the latest file as a new version of the original file to the TDP.