Tetra File-Log Agent Performance Guidelines

Tetra File-Log Agent performance depends on the Agent’s deployment environment and the amount of data being processed. This topic provides key performance metrics for the Tetra File-Log Agent and describes the factors that influence those metrics. It also provides baseline performance testing results that you can use to help optimize the Tetra File-Log Agent’s performance in your environment.

For information about monitoring a Tetra File-Log Agent’s performance, see Monitor Tetra File-Log Agent Progress and Metrics.

📘

NOTE

For help optimizing your Tetra File-Log Agent’s performance, contact your customer success manager (CSM).

Key Performance Metrics

Tetra File-Log Agent performance is typically measured by using the following performance metrics:

  • Scan frequency: How frequently the Agent scans file or folder paths.
  • Scan duration: How long it takes the Agent to scan file or folder paths.
  • Number of files uploaded: How many files the Agent uploads to the TDP in a given time period.
  • File latency: How long it takes a file to be uploaded to the TDP after it’s created or modified.

Influencing Factors

Scan frequency is configured through the File Watcher Service. Scan duration, the number of files uploaded, and file latency are determined by many factors, including the following:

  • The number of scanned paths
  • Connection status between the Agent and the Tetra Data Platform (TDP)
  • Available disk space on the host server
  • Disk usage
  • CPU usage
  • Memory usage

📘

NOTE

If file system events are supported in your environment, file latency drastically improves because the Agent doesn’t need to scan all of the files in each path to detect changes. For more information, see File Systems and Protocols.

Tetra File-Log Agent Performance Testing

TetraScience conducts performance testing of the Tetra File-Log Agent with each release. Two tests are used to simulate different scenarios: One for historical data loads, and another for continuous data scanning.

📘

NOTE

The following tables show guidelines for how a Tetra File-Log Agent performs on a host server with four logical processors, 16 GB of memory, and a 10 Gbps network connection.

Historical Data Loads

The following test simulated configuring a Tetra File-Log Agent with new paths, or adding a large number of new files into monitored paths.

Test

  • Monitored paths: 25
  • Number of files scanned: 100,000 (4,000 files per path at 3 MB each)

Results

Key Performance MetricPerformance Test Result
Scan frequency19 seconds
Scan duration24 seconds
Number of files uploaded100,000
File latency- 30 seconds for the first scanned files
- 2.5 hours for the full batch of 100,000 files

Continuous Data Scanning

The following test simulated a Tetra-File Log Agent with a large database (150 GB+) that scans more than 10 million files, which are generated consistently over a three-day time period.

Test

  • Monitored paths: 100
  • Number of files scanned: 10 million (17 generated every minute)

Results

Key Performance MetricPerformance Test Result
Scan frequency20 minutes
Scan duration20 minutes
Number of files uploaded70,000+
(Mean) File latency39.5 seconds