Version

This guide is for version 2.3.x of the software.

📘

NOTE:

The File-Log Agent can only retrieve files stored on the NTFS file system. Other file systems are not supported.

Installation

TetraScience provides an installation package (.msi). The user needs to run the installation to set up the Agent on to the server. The default installation path is C:\Tetrascience\Tetrascience.Agent.File-Log.v2.x.x.

Configuration

Starting from version 2.3.1, the File Log Agent provides a Windows-based Management Console for easy configuration.

1. File-Log Group User It is the service account running the Agent.

• If the account is left as blank, the Agent will run using Windows predefined LocalSystem account.

• If the Agent needs to monitor the network shared folders, it is suggested to provide a domain Service Account which should fulfill the following requirements:

  • The service account has at least read permission of the network shared folders (including the subfolders and files contained in those folders)
  • The service account is part of the local user group of the host server
  • The service account needs the Log on as service right

2. File-Log Agent Job time interval It defines how often the Agent internal Jobs run.

• Status Check Notify TetraScience Platform that the Agent is up and running
• Batch Interval Time interval for the Agent to upload the files (used by fileWatcher service)
• Stream Interval Time Interval for the Agent to upload the new content of log files (used by logWatcher service)
• Agent Log Time Interval for the Agent to upload Agent Logs

3. Agent Configuration

• Data Connection Option. TetraScience provides two options for Agent to integrate with TetraScience Platform, Generic Data Connector and User Defined Integration. If you are using a Data Hub with a Generic Data Connector, select Generic Data Connector option. If you want the agent to talk to the TetraScience Data Integration Platform directly, select User Defined Integration option.
• Agent Id. The Agent can be set up from either Generic data Connector or User Defined Integration. The Agent Id can be retrieved after the setup completes.
• Data Connector. The base URL of Generic Data Connector or User-Defined Integration. If you are connecting to a Generic Data Connector, please specify the URL with the port number. For example, https://10.10.10.10:8082
• TertraScience API Key. The key used by the Agent to provide as part of the request header when uploading the files. It is used by User-Defined Integration only.

Please follow the links of Generic Data Connector and User-Defined Integration above to get more detailed information.

4. Enable the Service The user can enable either of the service or both.

📘

NOTE:

The File-Log Agent can only retrieve files stored on the NTFS file system. Other file systems are not supported.

FileWatcher Service Setting

When the FileWatcher Service is enabled, the user can configure the service by providing the path to monitor the file or folder, file source type, glob pattern, file metadata, and tags, etc.

Interval In Seconds Specify the time span for the Agent to determine if the file should be uploaded by comparing the file's Last Write Time

Start Date The Agent ignores files/folders older than the selected date

Path The user can specify which folder path to monitor, the glob pattern to select the files/folders, the additional information, such as source type, meta data and tgs, which should be associated with the file when uploading the TetraScience Platform. The user can add new path, delete a path or edit an existing path.

Among the data fields used by FileWatcher service, the Path, Patterns, and File Watcher Mode are mandatory. If Source Type is left as empty, the Agent will use unknown as default.

Starting from v3.0.0, if the user updates the source type, Metadata, or tags onto the folder path which already been processed, the Agent will reupload the files to TetraScience Data Platform by attaching the updated Source Type, Metadata or Tags. The detail can be found in FAQ - 7. How the Agent behaves if Source Type, Metadata or Tag is changed?

LogWatcher Service Setting

When the LogWatcher Service is enabled, the user can configure the service by providing the path to monitor the file or folder, file source type, glob pattern, file metadata, and tags, etc.

Interval in Seconds It specifies how often the LogWatcher Service process to gather the file's metadata, determine the changes, and generate the new content if the files are updated.

Output Folder It specifies the local folder where the new content is stored initially before upload. That folder will be automatically created by the Agent if it doesn't exist.

Archive Folder It specifies the local folder where the new content is stored after upload. That folder will be automatically created by the Agent if it doesn't exist.

Start Date The Agent ignores files/folders older than the selected date

Path The user can specify which folder to monitor, the glob pattern to filter the files, the additional information, such as source type, metadata, and tags, should be associated with the file when uploading the TetraScience Platform. The user can add a new path, delete a path, or edit an existing path.

Among the data fields used by FileWatcher service, the Path, Patterns, and File Watcher Mode are mandatory. If Source Type is left as empty, the Agent will use unknown as default.

Starting from v3.0.0, if the user updates the source type, Metadata, or tags onto the folder path which already been processed, the Agent will reupload the files to TetraScience Data Platform by attaching the updated Source Type, Metadata or Tags. The detail can be found in FAQ - 7. How the Agent behaves if Source Type, Metadata or Tag is changed?

Start the Agent

When the configuration is complete, the user needs to Save the changes, and click the Start button to start the Agent.

🚧

NOTE

Once installation and your initial configuration are complete, use a windows task script to start the agent daily (such as at 1:00 a.m.) to ensure that the agent continues to remain running. For information on how to do this, view this topic.

Questions or Issues

If running into any issue, please check Common Issues and Solutions section.

If you like to learn more about TetraScience File-Log Agent, please read FAQ section or reach TetraScience directly.