API DocumentationProduct Documentation

Tetra Empower Agent v5.2.x and Tetra Data Schema Release Notes

Suggest Edits

The following are the Tetra Empower Agent release notes for versions 5.2.x.

🚧

IMPORTANT

For the Tetra Empower Agent v5.2.x releases, all of the associated Tetra Data components must be updated together. These components include the intermediate data schema (IDS), protocol, task script, and agent. Any prior versions of the IDS (v13 and earlier) will be incompatible with Tetra Empower Agent v5.1.0 and higher.

v5.2.2

Release date: 1 April 2024 (Last updated: 24 April 2024)

What's New

TetraScience has released its next version of the Tetra Empower Agent, version 5.2.2. This release focuses on resolving several known issues and provides new functionalities for its associated Tetra Data components.

Prerequisites

The following Tetra Data components are required for the Tetra Empower Agent v5.2.2:

  • Tetra Data
    • lcuv-empower IDS v16.0.0 (recommended) or v15.0.0
      • empower-raw-to-ids Protocol v9.0.0 (recommended) or v8.2.0
      • Task Script v15.0.0 (recommended) or v14.1.0

The following Tetra Data components are compatible with Tetra Empower Agent v5.2.2:

  • empower-audit-trail IDS v3.0.0
    • empower-audit-trail-raw-to-ids Protocol v3.0.0
    • Task Script v3.0.0
  • empower-project-user IDS v2.0.0
    • empower-project-user-raw-to-ids Protocol v2.1.0
    • Task Script v2.1.0
  • empower-message-center IDS v2.0.0
    • empower-message-center-raw-to-ids Protocol v2.2.0
    • Task Script v2.2.0

New Functionality

New Functionality includes features not previously available in the Tetra Empower Agent. These features may or may not affect Intended Use for GxP validation purposes.

Tetra Data

LCUV-Empower

  • Added a save_all_datacubes_to_parquet config parameter to control if either all or only 3D MS datacubes are output to Apache Parquet files

    📘

    NOTE

    There is a limitation on the size of the data that can be saved to the Parquet file. During testing, files of approximately 700 MB have failed to generate Parquet files when the saveAllDatacubesToParquet value is set to true. If you are processing a large file, it's recommended that you set this config parameter to false to avoid Parquet file generation failure.

  • Added parsing of top level sample_sets[*] to capture all sample sets (Process only and acquired) in a given raw JSON input
  • Added warning when the datacubes[*].measures[*].unit parameter is null
  • Updated parsing of methods[*].processing.update_pet_type to methods[*].processing.update_ret_type
  • Updated elasticsearch.json to include the nested field methods[*].calibration.average_cal_points[*]

Enhancements

Enhancements are modifications to existing functionality that improve performance or usability, but don't alter the function or intended use of the system.

  • There are no new enhancements in this release.

Bug Fixes

  • Performance of the Injection Summary tab has been improved. Projects are now selected using a dialog and supports tens of thousands of projects and the injection summary and error grids now support millions of injections.
  • Deferred, high priority injections will not block the generation of normal priority injections.
  • Fix for issue causing multiple scans of the same project to occur in separate processes in parallel. This potentially resulted in duplicate records in the agent database for some injections.
  • !QuickSet is recognized as a valid Sample Set Method. The Agent can process the injection even when the option Continue Generation on ToolKit Error is set to No.
  • To avoid unnecessary generation errors, the Agent no longer fetches the calibration when the calibration ID is 0.

Deprecated Features

There are no new deprecated features in this release. For more information about Tetra Data Platform (TDP) deprecations, see Tetra Product Deprecation Notices.

Known and Possible Issues

  • When the agent generates RAW JSONs, the agent uses the Sample Set Method name to include the sample set method information in the RAW JSON. This is the only toolkit method available. Customers might have changed the Sample Set Method. If you're using the Sample Set Method in the RAW JSON or IDS JSON, it's recommended that customers compare the SampleSet.SampleSetMethodDate in the RAW JSON and the sample.set.method.created_at in the IDS JSON with the time of the injection to confirm which Sample Set Method is referred to.
  • If the Empower project name has non-ASCII characters (for example, accents or dieresis), the order of projects displayed in the Empower project browser and agent management console may not be the same.
  • Sample Set Method information for an Injection through the toolkit provides the Sample Set Method name of the approved Sample Set Method version instead of the Sample Set Method ID of the version used to set up the sample set. This may or may not be the Sample Set Method version that was actually used for the sample set in the Injection.
  • The Agent will not ingest the file if the injection contains the results processed using GPC/GPCV (Gel Permeation Chromatography) processing methods

Upgrade Considerations

After customers complete the upgrade to Tetra Empower Agent v5.2.2 and open the management console for the first time, the console prompts them to run a database upgrade script. This script is required to clean up potential duplicate injections.

To upgrade the agent, see Common Tetra Agent Upgrade and Installation Information.

v5.2.1

Release date: 12 January 2024 (Last updated: 24 April 2024)

What's New

TetraScience has released its next version of the Tetra Empower Agent, version 5.2.1. This release focuses on resolving several known issues. There are no new functionalities or enhancements in this release.

Prerequisites

The following Tetra Data components are required for the Tetra Empower Agent v5.2.1:

  • Tetra Data
    • lcuv-empower IDS v16.0.0 (recommended) or v15.0.0
      • empower-raw-to-ids Protocol v9.0.0 (recommended) or v8.2.0
      • Task Script v15.0.0 (recommended) or v14.1.0

The following Tetra Data components are compatible with Tetra Empower Agent v5.2.1:

  • empower-audit-trail IDS v3.0.0
    • empower-audit-trail-raw-to-ids Protocol v3.0.0
    • Task Script v3.0.0
  • empower-project-user IDS v2.0.0
    • empower-project-user-raw-to-ids Protocol v2.1.0
    • Task Script v2.1.0
  • empower-message-center IDS v2.0.0
    • empower-message-center-raw-to-ids Protocol v2.2.0
    • Task Script v2.2.0

New Functionality

New Functionality includes features not previously available in the Tetra Empower Agent. These features may or may not affect Intended Use for GxP validation purposes.

  • There is no new functionality in this release.

Enhancements

Enhancements are modifications to existing functionality that improve performance or usability, but don't alter the function or intended use of the system.

  • There are no new enhancements in this release.

Bug Fixes

  • When the Agent fetches Channel data and gets a non-Toolkit exception such as an Out of Memory error, the Injection RAW file generation now fails rather than creating an empty Channel data cube.
  • Improvements to how the Agent generates data cubes reduces the chances of Out of Memory errors during Injection generation.

Deprecated Features

There are no new deprecated features in this release. For more information about Tetra Data Platform (TDP) deprecations, see Tetra Product Deprecation Notices.

Known and Possible Issues

  • When the agent generates RAW JSONs, the agent uses the Sample Set Method name to include the sample set method information in the RAW JSON. This is the only toolkit method available. Customers might have changed the Sample Set Method. If you're using the Sample Set Method in the RAW JSON or IDS JSON, it's recommended that customers compare the SampleSet.SampleSetMethodDate in the RAW JSON and the sample.set.method.created_at in the IDS JSON with the time of the injection to confirm which Sample Set Method is referred to.
  • If the Empower project name has non-ASCII characters (for example, accents or dieresis), the order of projects displayed in the Empower project browser and agent management console may not be the same.
  • Sample Set Method information for an Injection through the toolkit provides the Sample Set Method name of the approved Sample Set Method version instead of the Sample Set Method ID of the version used to set up the sample set. This may or may not be the Sample Set Method version that was actually used for the sample set in the Injection.

Upgrade Considerations

After customers complete the upgrade to Tetra Empower Agent v5.2.1 and open the management console for the first time, the console prompts them to run a database upgrade script. This script is required to implement the improvements in data cube generation.

To upgrade the agent, see Common Tetra Agent Upgrade and Installation Information.

v5.2.0

Release date: 20 December 2023 (Last updated: 24 April 2024)

🚧

IMPORTANT

For the Tetra Empower Agent v5.2.0 release, all of the associated Tetra Data components must be updated together. These components include the intermediate data schema (IDS), protocol, task script, and agent. Any prior versions of the IDS (v13 and earlier) will be incompatible with Tetra Empower Agent v5.1.0 and higher.

TetraScience has released its next version of the Tetra Empower Agent, version 5.2.0. This release focuses on reducing data latency and improving the Agent’s performance and usability by introducing new functionalities and enhancements, such as the following:

  • Multiple Agents can now connect to the same Empower Server, which allows the system to scale horizontally
  • Ability to set Scanning Priority for projects
  • Shorter delays when the Agent loses connectivity
  • Ability to configure the maximum number of projects included during the scan process (set with a default value of 20 that helps optimize performance and reliability)

This release also includes several bug fixes for scanning, injection generation, and project-user access permissions.

Here are the details for what’s new in Tetra Empower Agent v5.2.0.

Prerequisites

📘

NOTE

Tetra Empower Agent release notes will now include the recommended protocol version (task script and IDS) that is compatible with each version of the Empower Agent.

The following Tetra Data components are required for the Tetra Empower Agent v5.2.0:

  • Tetra Data
    • lcuv-empower IDS v16.0.0 (recommended) or v15.0.0
      • empower-raw-to-ids Protocol v9.0.0 (recommended) or v8.2.0
      • Task Script v15.0.0 (recommended) or v14.1.0

The following Tetra Data components are compatible with Tetra Empower Agent v5.2.0:

  • empower-audit-trail IDS v3.0.0
    • empower-audit-trail-raw-to-ids Protocol v3.0.0
    • Task Script v3.0.0
  • empower-project-user IDS v2.0.0
    • empower-project-user-raw-to-ids Protocol v2.1.0
    • Task Script v2.1.0
  • empower-message-center IDS v2.0.0
    • empower-message-center-raw-to-ids Protocol v2.2.0
    • Task Script v2.2.0

New Functionality

New Functionality includes features not previously available in the TDP or Tetra Empower Agent. These features may or may not affect "intended use" for GxP validation purposes.

Tetra Data

LCUV-Empower

  • Added config parameter save_all_datacubes_to_parquet to control whether to output all or only 3D MS datacubes in a parquet file
    • There is a limitation on the size of the data to be saved to the Parquet file. Based on our experience, files of approximately 700MB have failed to generate Parquet files when the saveAllDatacubesToParquet configuration is set to true. Therefore, if you are processing a large file, we recommend setting saveAllDatacubesToParquet to false to reduce the data size that goes into the Parquet file
  • Added parsing of top level sample_sets[*] to capture all sample sets (Process only and acquired) in a given raw JSON input
  • Added warning when datacubes[*].measures[*].unit is null
  • Updated parsing of methods[*].processing.update_pet_type to methods[*].processing.update_ret_type
  • Updated elasticsearch.json to include the nested field methods[*].calibration.average_cal_points[*]

Empower-Audit-Trail

  • Removed log_entries as a nested field in Elasticsearch
  • Added support for Empower Agent v5.x and higher

Empower-Project-User

  • Added support for Empower Agent v5.x and higher
  • Added system.agent

Tetra Empower Agent

Multiple Agents Can Now Upload Data to the Same Location

Customers can now configure multiple Agents to upload data to the same Amazon Simple Storage Service (Amazon S3) bucket within the TDP by specifying a Destination Id on the Connection Configuration page of the Agent Management Console.

New Scanning Priority Settings for Projects

Empower projects can now be assigned a Scanning Priority in the Agent Management Console: High or Normal. A separate process runs the High priority scans and there are fewer High priority projects thus reducing scan latency.

📘

NOTE

Setting Scanning Priority at the project level overrides the Agent's time-based, automatic prioritization settings.

Empower Server Connectivity Is Now Monitored

The Agent’s connectivity to the Empower server is now monitored. If connectivity is lost, the Agent also now automatically terminates any scan and generation processes that are running, so they no longer need to wait for a timeout to occur first.

To deactivate this functionality, customers can turn off the Enable Empower Connection Status Check toggle in the Advanced Settings dialog in the Agent Management Console.

Enhancements

Enhancements are modifications to existing functionality that improve performance or usability, but don't alter the function or intended use of the system.

Tetra Empower Agent

New Max Number of Projects per Scan Process Setting

A new Max Number of Projects per Scan Process field in the Advanced Settings dialog in the Agent Management Console gives customers the ability to configure the maximum number of projects that are scanned by a single scan process. The default value is set to 20 to help optimize both performance and reliability.

📘

NOTE

Decreasing the default Max Number of Projects per Scan Process value can help maintain a more strict priority ordering, if desired. Including a smaller number of projects in each scan can help reduce the risk of delaying the scan of later projects within the batch when the scan encounters an error.

Local States for the Project/User Access Permission Non-Experiment Service Can Now Be Cleared

Customers can now clear the local state for the Agent’s Project/User Access Permission Non-Experiment service. Clearing this local state causes a complete scan and generation of Project / User access permissions, rather than an incremental generation, and can be helpful when the service’s local state is no longer valid. (Issue #2854)

To clear the local state, customers can now select the Clear Cache button on the Injection Service - Non-Experiment Summary - Project/User Access Permission tab in the Agent Management Console.

Additional Enhancements

Logging improvements for errors that occur when fetching projects from Empower during generation.

Bug Fixes

The following bugs are now fixed:

Tetra Data

Empower-Audit-Trail

  • Fixed parsing of long timestamp format

Tetra Empower Agent

Scanning Process Bug Fixes

  • Out of memory (OOM) exceptions are now handled during scanning, and the Agent terminates the scan process, rather than trying to scan the remaining projects in each batch. The remaining projects in the batch are now added as a separate batch for later scanning.
  • Unrecoverable errors are now handled during priority scanning. The Agent then terminates the scan process immediately.
  • Empower Software Development Kit (SDK) errors that are unrecoverable (those error codes ending in 105) are now handled. The Agent now fetches result objects and immediately terminates the scan process. During testing, these errors have proven to persist until the process ends and a new process is launched.
  • A separate, configurable timeout and batch size for project priority scanning is now available in the Advanced Settings dialog of the Agent Management Console.

Generation Process Bug Fixes

  • Access to the channel data files when retrieving raw data is now validated by the Agent. Generation now fails if the data file isn't accessible, because the SDK returns without error when fetching 2D channel data and returns an empty data cube.
  • Corrupted state (error code 0xc0000005) and OOM exceptions that were previously unhandled during scanning and generation are now handled and have improved logging. (Issue #2649)
  • Injection results are now fetched by using paging to avoid issues when a single injection has a large number of results.
  • Injections that fail generation because they exceed the generation timeout are now retried with a longer timeout.
  • For Empower 3 pre SR3, the Agent now ingests the injections immediately if injection data acquisition is complete. If the data acquisition isn't complete, the Agent will ingest the injection after the maximum number of acquisition hours that are configured. Previously, the Agent always waited for the maximum number of acquisition hours before ingestion.

Project / User Access Permission Process Bug Fixes

  • The Agent can now handle and recover from being in an Invalid state in the local store when the state was caused by errors generated when retrieving Project/User Access Permission non-experiment data.

📘

NOTE

After upgrading to Tetra Empower Agent v5.2.0, any local state stored for the Project/User Access Permission non-experiment data that’s not valid will be removed so that it can be regenerated.

  • Optimized Agent processing of Project / User Access Permissions to avoid unnecessary processing. The output remains unchanged.
  • The non-experiment generation process timeout is now separately configurable in the Advanced Settings dialog of the Agent Management Console by using the NonExperiment console Timeout (minutes) field.
  • Projects that aren't accessible by the Empower user account that the Agent is using are no longer scanned by the Project / User Access Permission scanning process. Because of this update, inaccessible projects will no longer cause error logs during scans.

User Interface Bug Fixes

The number of pending uploads reported in the Agent Management Console now displays accurately.

Deprecated Features

There are no new deprecated features in this release. For more information about TDP deprecations, see Tetra Product Deprecation Notices.

Known and Possible Issues

The following are known and possible issues that were previously identified:

Tetra Empower Agent

  • When the agent generates RAW JSONs, the agent uses the Sample Set Method name to include the sample set method information in the RAW JSON. This is the only toolkit method available. Customers might have changed the Sample Set Method. If you're using the Sample Set Method in the RAW JSON or IDS JSON, it's recommended that customers compare the SampleSet.SampleSetMethodDate in the RAW JSON and the sample.set.method.created_at in the IDS JSON with the time of the injection to confirm which Sample Set Method is referred to.
  • If the Empower project name has non-ASCII characters (for example, accents or dieresis), the order of projects displayed in the Empower project browser and agent management console may not be the same.
  • Sample Set Method information for an Injection through the toolkit provides the Sample Set Method name of the approved Sample Set Method version instead of the Sample Set Method ID of the version used to set up the sample set. This may or may not be the Sample Set Method version that was actually used for the sample set in the Injection.
  • When the Agent fetches Channel data and gets a non-Toolkit exception such as an Out of Memory error, the Injection RAW file it generates has an empty Channel data cube.

Operating Suggestions

  • If you're not interacting with the agent management console frequently, it's recommended to start the agent, close the management console, and then let the agent run in the background.
  • If the Empower server or client is under maintenance, it's recommended to stop the Tetra Empower Agent. Remember to restart the agent when Empower is back online. The unfinished tasks will be resumed by the agent once it is started again.
  • If the Empower Oracle service is stopped, then stop the agent and restart the agent after the Oracle service is back online.

Upgrade Considerations

To upgrade the agent, see Common Tetra Agent Upgrade and Installation Information. To update the protocol, see Edit a Pipeline.

This version is compatible as a direct upgrade from Tetra Empower Agent v5.1.0. During the upgrade, keep in mind the following:

  • The upgrade for the Tetra Empower Agent v5.1.0 and Tetra Data Schema v16.0.0 release is NOT backward-compatible. This means the RAW files generated by this agent cannot be processed by previous protocol versions. The agent is now also able to generate 3D MS data cubes with correct data precision.
  • Performance-wise, the new protocol’s runtime will be longer when compared to the previous protocol versions because of the IDS structure change and the newly added fields. From our internal testing, customers can expect an increase in runtime (tens of seconds) for Empower data without 3D Mass Spec data. For Empower data with 3D Mass Spec data, customers can expect a longer runtime because of the newly added Parquet file creation for 3D MS data. The time increase will be dependent on the size of the 3D MS data. The larger the data, the more time it will take to generate the Parquet file.
  • If data generated by previous agent version is not available for re-upload, empower-raw-to-ids protocol v8.2.0 allows for migration of previous agent versions to IDS v15.0.0. Only data available in the RAW JSON from the previous agent versions will be populated in the IDS. Data not available is populated as null values.
  • If 3D MS data precision is required, configure the 3D MS Method as GetSpectra for the specific projects in the Project grid, save the changes, and start the agent. Keep in mind that 3D MS data cubes generation using GetSpectra Toolkit method is significantly slower than generating 3D MS data cubes with the Get3D Toolkit method and will increase the generation time for each injection.

Other Release Notes

To view other Tetra Agent and Tetra Data release notes, see Tetra Agent and Tetra Data Release Notes.

To view other TDP release notes, see Tetra Data Platform (TDP) Release Notes.

Updated 3 days ago