Tetra File-Log Agent Release Notes (v4.2.3 and Below)

📘

NOTE

As newer versions of Agents are released, their release notes will appear here.

This document provides the Tetra File-Log Agent release notes. Release notes are organized by version number.

📘

Upgrade Instructions

To upgrade the Tetra File-Log Agent, see this document.

v4.2.3

RELEASE DATE: 11/16/2022

Enhancements

There are no enhancements for this release.

Bug Fixes

The bug fixes for this release appear below.

• AG-2579: FLA Uploads stop after several thousand using GDC/L7 Proxy

This resolves a potential hanging issue when using the GDC with L7 proxy in some workloads and environments. Versions 4.1.x through 4.2.2 are using the asynchronous AWS SDK methods and a customer has reported a scenario of the upload hanging when using GDC and L7 proxy on a single instance. Based on internal testing, we’ve found the synchronous AWS SDK methods to be more reliable for our use case and this release reverts to using those.

• AG-2612: Memory leak when scanning path using recursive glob pattern ("**/")

The memory leak is due to task cancellation tokens were not disposed properly in the path scanning job.

📘

Note

This is not a critical patch and may only be necessary when experiencing this specific scenario.

Minor Functionality Changes

There are no minor functionality changes for this release.

v4.2.2

RELEASE DATE: 9/23/2022

🚧

Critical Patch

v4.2.2 is a critical patch. TetraScience strongly recommends that you upgrade to v4.2.2 if you have already upgraded to v4.2.x.

📘

NOTE

The release note for v4.1.4, which has the same bug fix as v4.2.2, has also been released.

Enhancements

There are no enhancements for this release.

Bug Fixes

The bug fixes for this release appear below.

• AG-2367/AG-2396 Improves error handling and logging when re-initializing file system event listeners.

v4.1.4

RELEASE DATE: 9/23/2022

🚧

Critical Patch

v4.1.4 is a critical patch. TetraScience strongly recommends that you upgrade to v4.1.4 if you have already upgraded to v4.1.x.

Enhancements

There are no enhancements for this release.

Bug Fixes

The bug fixes for this release appear below.

• AG-2367/AG-2396 Improves error handling and logging when re-initializing file system event listeners.

v4.2.1

RELEASE DATE: 9/1/2022

🚧

Critical Patch

v4.2.1 is a critical patch. TetraScience strongly recommends that you upgrade to v4.2.1 if you are using a previous version of the agent.

Enhancements

There are no enhancements for this version of the software.

Bug Fixes

AG-2338 - Paths configured with File Watch Mode “file” using a recursive glob pattern.
Any pattern starting with [**/], such as the following:

**/*.txt, **/*, or  **/*.*

It can get "hung" indefinitely scanning (but not actually doing anything) if some unhandled errors occur. These errors are correctly handled and logging improved to better diagnose and troubleshoot issues of this nature.

Deprecated Features

There are no deprecated features for this version of the software.

Minor Functionality Changes

There are no minor functionality changes for this version of the software.

v4.1.3

RELEASE DATE: 9/1/2022

🚧

Critical Patch

v4.1.3 is a critical patch. TetraScience strongly recommends that you upgrade to v4.1.3 if you have already upgraded to v4.1.x.

Enhancements

There are no enhancements for this version of the software.

Bug Fixes

AG-2338 - Paths configured with File Watch Mode “file” using a recursive glob pattern.
Any pattern starting with [**/], such as the following:

**/*.txt, **/*, or  **/*.*

It can get "hung" indefinitely scanning (but not actually doing anything) if some unhandled errors occur. These errors are correctly handled and logging improved to better diagnose and troubleshoot issues of this nature.

Deprecated Features

There are no deprecated features for this version of the software.

Minor Functionality Changes

There are no minor functionality changes for this version of the software.

v4.2.0

RELEASE DATE: 8/22/2022

Enhancements

Enhancements are new features and functionality that have been added to the software. Please check the section below to review the enhancements made for this version of this software:

AG-2104 - A file can be sent from TDP to the agent via a command service and the agent can receive the file and place it in a directory it can access. Refer to FLA Drop File Command to learn how to use this command service.

AG-1975/AG-2019/AG-2084 - The path watching/scanning algorithms and database performance have been optimized to improve the agent's scan performance. If you upgrade the Agent from previous version, it might experience some delay when starting the Agent.

Bug Fixes

There are no bug fixes for this version of the software.

Deprecated Features

There are no deprecated features for this version of the software.

Minor Functionality Changes

There are no minor functionality changes for this version of the software.

v4.1.2

RELEASE DATE: 6/30/2022

Enhancements

There are no enhancements for this version of the software.

Bug Fixes

The following bug fixes were made to this version of the agent.

• AG-2087 - When the agent is started, the scrollbar for the list of paths is no longer disabled. The scrollbar for the paths is enabled whether the agent is started or stopped.

Deprecated Features

Removed the LogWatcher functionality.

v4.1.1

RELEASE DATE: 6/8/2022

🚧

Critical Patch

v4.1.1 is a critical patch. TetraScience strongly recommends that you upgrade to v4.1.1 if you have already upgraded to v4.1.0.

Note also that the enhancements, bug fixes, and other changes found in v4.1.0 are in v4.1.1 as well.

Enhancements

No new features and functionality have been added to the software for this release.

Bug Fixes

Here are the bug fixes made for this version of the agent software:

• AG-2017 - Fixes a bug that was introduced in the Tetra File-Log Agent v4.1.0, which caused the Agent to use the S3 Direct Upload option, even if it was not selected. Problem was resolved and the files are now uploaded using the option that the user selects for file upload.

Deprecated Features

There are no deprecated features for this version of the software.

Minor Functionality Changes

There are no minor functionality changes for this version of the software.

v4.1.0

RELEASE DATE: 31 May 2022

📘

NOTE

Version 4.1.0 of the Tetra File-Log Agent is no longer available. Please upgrade to version 4.1.1 instead. All enhancements, bug fixes, and other changes are in version 4.1.1 of the software.

Enhancements

Enhancements are new features and functionality that have been added to the software.

• AG-1194/AG-1196/AG-1925: You can programmatically modify agent configuration using the API. (You can also modify the agent configuration via the user interface in the TDP.) New documentation has been produced that shows you how to use API endpoints to programmatically create and configure an agent. See the link to the documentation FLA Programmatic Agent Creation and Configuration here.
• AG-1158: There is now a more equal priority for the ingestion of files from all paths. For every path, the Agent uploads the newest files first. Files from other paths won’t be blocked by a single path.
• AG-1595: Global settings for the File Change Interval, Start Date, and Use Path Configurations in File-Log Agent Management Console have been removed. Also:
  • The File Change Interval and Start Date for each path are now displayed and must be filled in.
  • When the configuration is pushed from TDP to the agent or locally saved in the File-Log Agent Management Console, the Use Path Configurations is now always true.
  • If you are migrating file paths from a pre-FLA v4.1.0 version of the software:
    • If Use Path Configurations is false, the global File Change Interval and Start Date are copied to all path level information.
    • If Use Path Configurations is true, then there is no change.
• AG-978/AG-1004/AG-1005: Configuration changes, as well as when the agent is stopped and started are now recorded in the Agent User Audit Trail.
• AG-1177: When the FLA path configuration is changed locally or in the cloud, a copy of the path configuration is saved in the manual.json file, which is uploaded to TDP.
• AG-947: File Properties, such as file creation date, last modified date, and file size are sent with files when uploading to TDP. They become part of the metadata of the S3 object.
• AG-1584: The default folder path setting in the FileWatcher Service has been removed.
• Several enhancements were made to improve throughput, performance, validation, logging, consistency, and usability. These include, (but are not limited to):
  • AG-1198: Improving how the FLA configuration is updated.
  • AG-1262: Adding more information in the logs, such as the agent host machine's IP, agent's host machine name, and connection type (CDC or GDC).
  • AG-1440: Allowing log files larger than 1 MB to be uploaded Cloudwatch.
  • AG-1959: Changing “Enable Queue” label to "Receive Commands".
  • Improving caching, streamlining workflows, and improving performance.

Bug Fixes

The following bug fixes were made to this version of the agent.

• AG-1845: Changes to files monitored by folder mode paths are now detected by the Event monitoring.

Deprecated Features

• Removed the LogWatcher functionality.

Minor Functionality Changes

• Logs include more information. See the enhancements section of these release notes.
• AG-1949: Command response timestamps are now in UTC standard format.
• AG-1560: The File-Log Agent now uploads files up to 5 TB when you use the S3 Direct Upload option. Large files that are uploaded no longer automatically timeout after an hour.

v.4.0.5

RELEASE DATE: 22 April 2022

Enhancements

Enhancements are new features and functionality that have been added to the software. Click the arrows next to each section to review the enhancements made for this version of this software:

• AG-1750/AG-1795 We have improved FLA logs to help with troubleshooting. Logs now capture more information. Logs now include:
  • “changeDetectedBy” - how changes were detected, by an Event or by Scan - which is a task that the File-Log Agent software performs.
  • “fileWatchMode” - whether a file change was detected in file or folder mode.
  • “orgSlug” - the organizational identifier for your organization in TDP.
  • “message” - indicates when the folder path scanning starts and provides more details about what has been changed, such as the last write time, the length of the file or folder, and the file count. Additionally, permission issues are logged.

Bug Fixes

Click the arrows next to each section to review the bug fixes made for this version of the agent software:

• AG-1796 - If FLA does not have the permissions needed to process a file that is part of a folder, it now shows an error that is displayed on the Management Console’s Summary page. This will allow you to be aware that it was not processed so that you can address the issue.

Deprecated Features

The LogWatcher feature is on the deprecation path.

Minor Functionality Changes

Logs include more information. See the enhancements section of these release notes.

v.4.0.4

RELEASE DATE: 16 March 2022

🚧

Critical Patch

Version 4.0.4 is a critical patch. TetraScience strongly recommends that you upgrade to v4.0.4.

Enhancements (New Features and Functionality)

There are no new features and functionality for this release.

Bug Fixes

The following bug fixes are available for the Tetra File-Log Agent:

• AG-1432 The Start Date field in the File Log Watcher now works properly. The Start date field is documented here.
• AG-1433 The File Watcher now correctly processes files that are stored in different paths but that have the same file name.

Other Notes

This release includes the enhancements and fixes made to Patch 4.0.2 and 4.0.3.

v.4.0.3

RELEASE DATE: 17 January 2022

📘

Note

Patch 4.0.3 is no longer available. Please upgrade to v4.0.4 instead.

v4.0.2

RELEASE DATE: 21 December 2021

Enhancements (New Features and Functionality)

This version of the software has a number of performance improvements due to transitioning the Tetra File-Log Agent from a 32-bit to a 64-bit architecture, and a reduced footprint.

Bug Fixes

The following bug fix(es) were made to the Tetra File-Log Agent.

• AG-1209 - OOM while uploading system metrics
  This issue, which involved high memory usage, was resolved by performance improvements.

v4.0.1

RELEASE DATE: 3 August 2021

Bug Fixes

The following bug fix(es) were made to the agent.

• [AG-781] - FLA Agent uses S3 direct upload as default
  This bug caused the S3 Direct Load option to always be enabled, even if you tried to manually disable it in the File-Log Agent's Management Console. This issue was fixed and you can now disable the S3 Direct Load option.

v4.0.0

RELEASE DATE: 21 July 2021

Enhancements (New Features and Functionality)

The following enhancements were made in this version of the software.

• You can enable or disable the queue service. The queue service allows you to determine whether to receive a command from the TDP.
• If you choose to enable the service, the Agent can input the path configuration pushed from TDP in the Cloud Config.
• The end-to-end performance has improved 2x -3x. (It is measured the time from when the file is detected to the time file is uploaded to TDP).

Bug Fixes

The following bug fixes were made to the agent.

• [AG-548] The Agent is able to upload empty file using the S3 direct upload option
• [AG-531] The Agent installer can delete the installation folder from the previous version

Minor Functionality Change

• The Agent is able to upload Agent Log with a file size greater than 1 MB using the S3 direct upload option.

v3.5.0

RELEASE DATE: 17 May 2021

This topic provides information on enhancements, bug fixes, deprecated features, as well as minor functionality changes for version 3.5.0 of the Tetra File-Log Agent.

Enhancements (New Features and Functionality)

There was an enhancement in this version of the software.

• The Tetra File-Log Agent metadata and tag logic matches the latest TDP updates. For more information about TDP metadata updates, see the TDP 3.0.0 Release Notes.
• A new option has been added that can check the status of the Tetra File-Log Agent daily and restarts it if it has stopped. For more information about how to enable this option see this section of the Tetra File-Log Agent installation guide.

Bug Fixes

The following bug fix(es) were made to the agent.
AG-292 - Tetra File-Log Agent Windows Management UI Agent Log Upload Interval Is Bound to a Wrong Field.

The Agent Log files Uploads Every field is now bound to the correct field (Interval In Seconds).

Deprecated Features

The following feature(s) are currently deprecated.

• N/A

Minor Functionality Changes

The following minor functionality changes are in this version of the software.

• The Tetra File-Log Agent now starts even if the File Path is invalid. The status has been added near the file path to indicate whether the path is valid. The status is refreshed periodically (with each scan interval).

• When the agent shuts down or is backing up, "pending" records that have been scanned and not yet uploaded are deleted from the queue. (Please note that the records themselves are not deleted.) When the agent restarts again, those files will be rescanned and uploaded to the TDP. This solves an edge-case issue (AG-281).

• Source Type in the File Watcher section (Folder Path Configuration) is now mandatory. Previously if this was left blank, the value was "unknown". Note that for the Log Watcher section, Source Type is still optional.

Additional Notes
There are no other notes for this version of the software.

v3.4.0

New Features

• Users can re-upload the files with the file Last Write Date in a specific date-time range.
• The agent can now notify the user of the potential errors when starting or stopping the agent Windows Service application.

Bug Fixes

• The Agent is able to delete files with a read-only attribute from the local temp folder after the files are uploaded successfully to Tetra Data Platform.

Functionality Changes

• The agent won't trigger file reupload when the source type, metadata, or tag change on the file path.

• The agent converts the file path as the lower case letters when uploading the files to Tetra Data Platform (TDP).

• If you have previously installed a v3.x.x File Log Agent, after you upgrade to v3.4.0, note the following.

  • New file paths will all be converted to lowercase.
  • For already-scanned and uploaded files by v3.x.x File Log Agent, if there is a change to the last modified date or the file size, it will be uploaded with the original casing (the same casing the first time this file was uploaded).

v3.3.1

New Features

• Users can set up the file Start Date and File change interval on the individual file path level.

Enhancements

• Long file path (longer than 260 characters) for Windows 10, Windows Server 2016, and Windows Server 2019 can now be enabled. (Please check here to learn more
• The agent now notifies the user when s/he tries to close the Agent Windows Management Console UI without saving the changes.

Bug Fixes

• The search result of the file path should be case-insensitive when the Agent checks whether that file path is already under monitoring.

v3.2.1

Enhancements

• Agent doesn't require the user account configuring the Agent to have access permission to the folder paths. It delegates the folder path validation to the Windows Service part of the Agent.
• The Windows Management Console logs the detailed errors for diagnosis if the connection to the TDP fails.

v3.2.0

Enhancements

• Enhanced the internal algorithm to improve the performance of file change (new or updated) detection by 2-4 times.

Other Changes

• Adjusts the order of file upload to prioritize the latest updated files.
• Logs file access errors (e.g. file is moved or deleted) to Database. The Agent won't try to reprocess errored files in the later time interval.

v3.1.0

New Features and Functionality Changes

• The Agent can now run using a Service Account without Local Logon permission.

v3.0.0

New Features and Functionality Changes

• Upgraded to .NET Framework 4.8.
• Provides the option to upload the files directly to AWS S3 bucket. With this option turned on, the user is able to upload up to 500 GB file.
• Files can be reuploaded when the associated Souce Type, Metadata, or Tags are changed.
• Added Summary page to display the file processing status and the Agent hosting server performance metrics.
• Server performance metrics are now uploaded to AWS CloudWatch Metrics.
• Sqlite database is uploaded to AWS S3 bucket for disaster recovery. (Please note this feature requires turning on the S3 direct upload option.)
• No shortcut is added to user desktop, instead, a link is added to Windows Program menu.

Enhancements

• Preserves the case of file and folder name.
• Validates user account and permissions to the folder paths before the Agent started.

Other Changes

• Now uses data grid to manage Tags.

v2.3.2

New Features and Functionality Changes

• .NET Framework 4.7.2 is required.

Enhancements

• In LogWatcherService, the output files are removed after they are successfully uploaded to the Platform.
• Retains the Agent log files for 72 hours.
• Allow user to check the configuration setting for both LogWatcherService and FileWatcherService when the Agent is running.
• Popup Error notification message if File-Log Group User is invalid.
• Popup Notification messages if File-Log Group User doesn't have read permission of the paths. defined in the Filewatcherservice or LogWatcherService.
• The authentication errors are written to Agent Log files.

Bug Fixes

• Fixed Group User account is not used for running the File-Log Agent Service issue.
• Fixed System.Net.Http dll (4.2.0.0) reference issue.
• Fixed validation message when the pattern is missing from folder mode.
• Fixed NullReferenceException if JWT Token is not provided issue.

Installation Changes

• Applied Full Control permission to the installation folder for Machine\Users to avoid database permission issues.
• In migration, if the config.json from the previous version is incomplete, the Agent migrates the sections available in the config.json file.
• In migration, if the config.json from the previous version is invalid or missing, the Agent sets up configuration using default values.

Configuration Changes

• By default, FileWatcher is enabled and LogWatcher is disabled.
• Default agent ID and connector URL is blank.
• Default path is blank.
• All job time intervals are set as 10 seconds as the minimum.

v2.3.1

• Added a Windows-based Management Console UI to replace Config.json for Agent configuration.
• In the LogWatcher Service, grouped related output files into a subfolder in the output folder, to prevent potential file name collision.