Tetra LabX Tetra Data v5.0.0, Agent v2.0.x and Protocol v3.0.x Release Notes

📘

NOTE:

As newer versions of Agents are released, see Tetra Agent and Tetra Data Release Notes.

This document provides the Tetra LabX Tetra Data, Agent, and Protocol release notes. Release notes are organized by version number. The agents and protocols are compatible with Tetra Data v5.0.0.

📘

NOTE:

To upgrade the Tetra LabX Agent, see Common Tetra Agent Upgrade and Installation Information.

Agent v2.0.1/Protocol v3.0.0

Release Date: 11 May 2023 (Last updated: 20 March 2024)

What's New

TetraScience has released its next version of the Tetra LabX Agent, version 2.0.1. This release consists of bug fixes and enhancements, including making the agent more resilient and able to handle memory usage better. The agent is compatible with Tetra Data v5.0.0 and Protocol v3.0.0. For details of Protocol 3.0.0, see Protocol mettler-toledo-labx-raw-to-ids v3.0.0.

New Functionality

New functionality includes features not previously available in TDP. These features may or may not affect "intended use" for GxP validation purposes.

  • There’s no new functionality in this release.

Enhancements

Enhancements are modifications to existing functionality that improve performance or usability, but do not alter the function or intended use of the system.

  • Agent now has a one-hour timeout for the proxy used to access LabX. After one hour, the proxy is regenerated. This decreases the memory allocation and garbage collection for LabXHostService.
  • The ProductFamily field was added to the Instrument entity in RAW files. This field is present in LabX v11 onwards. If the agent is extracting data from LabX v10, this field will be null.
  • In the Summary tab of the Agent Management console, Total Generated has been replaced by Total Generated (Task files) and Total Generated (Audit trails) items. Total Generated (Task files) displays the number of task files that have been generated successfully. Total Generated (Audit trails) displays the number of audit trail files that have been generated successfully.
  • Agent hostname and IP are uploaded to TDP with agent heartbeat.
  • When fetching LabX audit trails, the agent has been optimized to improve performance and reduce in-memory allocation.

Bug Fixes

  • In previous versions of the agent, the manual configuration was not uploaded when the agent was started. This issue has been fixed and the manual configuration file is now uploaded to TDP when the agent starts.
  • In previous versions, the Agent Management console would become unresponsive when the Re-upload All the Task Files button was selected. This issue is fixed. A busy spinner may now momentarily appear when performing this operation. After that, the Agent Management console will remain active while the tasks are getting re-uploaded in the background.
  • In previous versions, when the agent is configured to run with a Local System account and the agent is started, the confirmation dialog with the message “No Agent Account User Name and Password, Click OK to continue install with Local system account” was not displayed. This issue is fixed, and the confirmation dialog is now displayed when a Local System account is used. Users can now choose OK to continue starting the agent, or choose Cancel to configure the agent with a Windows username and password.

Deprecated Features

There are no deprecated features in this release.

Known and Possible Issues

  • When the Tetra LabX Agent fetches data from LabX laboratory software versions 12 or earlier, the use of LabX SDK APIs (BasicIntegrationService and/or SystemIntegrationService) can lead to increased CPU and memory usage within LabX. METTLER TOLEDO implemented a fix for this issue in LabX v13. It's recommended that customers upgrade to LabX v13 to resolve the issue.

  • The agent extracts the ProductFamily field in the RAW JSON file, but protocol 3.0.0 doesn't parse this field into Tetra Data. A fix for this issue is in development and testing and is scheduled for a future Tetra LabX Agent release. If customers require the ProductFamily information from historical, already generated LabX task files in their Tetra Data, they can reprocess the files to include the information by using the Re-Upload All the Task Files feature in the Agent Management Console.

Upgrade Considerations

There are no upgrade considerations for this release.

Miscellaneous

  • In the Summary tab of the Agent Management console, the Pending to Generate, Total Generated (Task files), and Failed To Generate options refer to the number of task files that the agent will generate, or that it has successfully generated, or failed to generate, respectfully. The audit trails are accessed from database and only reported as Total Generated (Audit trails) as they do not have a pending or failed step.
  • The LabX Audit Trail generation feature in the agent is a beta feature. TetraScience will release a compatible protocol to generate Tetra Data from audit trail RAW JSON later.

Agent v2.0.0/Protocol v3.0.0

Release Date: 6 March 2023

New functionalities (including breaking change)

  • Updated Task RAW JSON to use task internal id, such as T32220, instead of task business Id, which is uuid. This is a breaking change from the v1 Tetra LabX Agent. The motivation for this change is to provide a more intuitive file name for Task RAW JSON, since the task internal id is displayed on the LabX UI, but not the task business id.

  • Introduced a beta service within Tetra LabX agent called: LabX Audit Trail service. It can periodically collect LabX audit trails from the LabX SQL server. This is requested by customers who want to analyze the user activities in LabX for better data analytics.

  • Only output “Completed“ and “Stopped/Aborted“ tasks. Tetra LabX Agent will no longer output “Paused“ tasks. This decision is made in collaboration with customers to minimize the task

Performance related

  • Limit the change detection to the tasks created in the most recent 14 days (user configurable), this can reduce the delay in terms of detecting new tasks and also reduce the load on the LabX host service. Customers have been reporting high memory usage on LabX host machine during testing, though we can not completely identify the root cause of high memory usage, we decide to proactively reduce the load on LabX server.

  • Reduce the amount of LabX audit trails generated during the operation of Tetra LabX agent by reusing the connection to LabX Server.

Supportability related

  • Add the ability to reupload LabX task RAW JSONs

  • Support the concept of LabX Name such that user can modify the file path of uploaded Task RAW JSON. Please be careful when using this configuration, since the LabX Name will be used in the Task RAW JSON's filePath in the TDP.

  • Upload Tetra LabX agent’s user action audit trail and LabX agent’s configuration when agent is started/stopped/changed

  • Add LabX connection test button

  • Add agent version and RAW JSON version to Task RAW JSON files

Data harmonization related

  • Handled the edge case instrument gets disconnected while performing the task and LabX returns multiple different instrumentVersionId for a single Task

  • Changed the naming of the IDS JSON to 0.json

Known and possible issues

  • LabX Task Recalculate
    • According to Mettler Toledo’s feedback, results recalculation is only available to UV and Titrator instruments. Tetra LabX agent can not detect results recalculation.
      We get the task time stamp by calculating the most recent date for Task creation date, Task result creation dates, Sample result creation dates to determine if a task is updated or changed. If the task is re-calculated, the creation dates do not change.
      We try to use LabX callback function to track all the changes, but the callback function does not have recalculate events either.
    • In LabX, modifying the sample size in a result set, that has been completed, is recorded in the audit trail. However, results of downstream calculations cannot be retrieved by the agent because LaBX API does not have an event to track this.
  • Direct Database Access Limitation

We read the audit trail records from LabX SQL database and it was tested on LabX V9, v10, v11.
Direct SQL access is not officially supported by Mettler Toledo. If the database schema is changed for any newer/older versions, the audit trail records might not be loaded properly by Tetra LabX Agent.

  • Result data fields missing from LabX API

You may find that certain data fields available in the LabX UI are not available in the IDS JSON.
This is because those fields are not available from LabX Web API.
TetraScience is open to getting these data fields from the LabX SQL database and adding to future releases.
Here are some fields our customers have confirmed to be missing:

  • Gross weight

  • Tare weight

  • Blocking state

  • Tolerance profile

  • Calibration

  • Agent and LabX Host Server Need to be in the Same Time Zone

Right now the LabX agent and the LabX host server need to be in the same time zone, otherwise the LabX agent does not know how to interpret the timestamp returned by the LabX server.
In future versions of Tetra LabX Agent, we plan to support users to deploy LabX server and LabX agent in different time zones by acquiring the timestamp through SQL Database directly.

  • Deleted Result Limitation

When an instrument is disconnected during an experiment, some of the results are labelled as “Deleted“, those data marks are set to 13 (see the screenshot below). Data mark 13 is for “measurement(s) deleted“. Those deleted results do not return from the LabX API and are not included in the RAW JSON and IDS JSON files.

  • Tetra LabX Agent Audit Trail Service does not support Windows Authentication
    Windows authentication allows a user to log into a computer and then access SQL server without asking for the password again. However, this is not supported by the current version of Audit Trail Service. It requires a valid SQL user to be entered on the Tetra LabX agent’s windows management console.

Protocol mettler-toledo-labx-raw-to-ids v3.0.0

With the update to LabX Agent new protocol is also being released to new handle the new features. Protocol common/mettler-toledo-labx-raw-to-ids/v3.0.0 will support LabX Agent 2.0. This protocol will run task script common/mettler-toledo-labx-raw-to-ids/v5.0.0 and will parse into IDS common/mettler-toledo-labx/v5.0.0.

Task script

Enhancements

  • Update systems to add new entry for each unique Instrument object

  • Add systems[*].pk_systems with a unique UUID for each system

  • Update users to allow for multiple users to be present from a single LabX export

  • Add users[*].pk_users with a unique UUID for each user

  • Update to use users[*].id and users[*].name instead of users[*].logon_name and users[*].full_name

  • Update results[*].samples[*].results[*] to follow the conventions of the Parameter IDS object

  • Update methods[*].metadata.scopes to handle multiple scopes in Method.MethodMetaData.Scopes

  • Remove systems[*].software.id

  • Update systems[*].software.name to parse from LabXName field

  • Update systems[*].software.version to parse from LabXVersion field

  • Add samples[*].pk_sample with a unique UUID for each sample

  • Add samples[*].id for SampleId1

  • Add samples[*].labels[*].name, samples[*].labels[*].value to hold SampleId2 - SampleId6

  • Remove samples[*].@link, samples[*].ids[*].key, samples[*].ids[*].value

  • Update users to store all unique User data instances from TaskResultSet.Samples[*].Results and TaskResultSet.TaskResults

  • Update systems to store all unique system/instrument instances from TaskResultSet.Samples[*].Results and TaskResultSet.TaskResults

  • Remove systems[*].type mapping as TaskResultSet.Samples[*].Results[*].Instrument.InstrumentModel is not an instrument type

  • Add the following fields under methods[*].metadata.parameters[*]

    • string_value

    • numerical_value

    • boolean_value

    • value_data_type

    • function_name

    • function_property_name

  • Add the following fields under methods[*].metadata.scopes[*].method_parameters[*]

    • string_value

    • numerical_value

    • boolean_value

    • value_data_type

  • Remove the following fields under methods[*].metadata.parameters[*] and methods[*].metadata.scopes[*].method_parameters[*]

    • max_length

    • min_length

    • decimal_places

    • value

  • Remove results[*].id

Assumptions

  • Raw file has Method, TaskResultSet sections

  • Instrument can appear multiple times in one data file like TaskResultSet[*].Samples[*].Results[*].Instrument or TaskResultSet[*].TaskResults[*].Instrument. The way we determine if the instrument entry is unique is by checking all the metadata in the instrument object.
    Same uniqueness check applied on instruments is applied on samples and users too.

  • If there is no fk_samples field defined in an entry in results, that means the entry is a "Task Result" not a "Sample Result".

IDS

Enhancements

To make queries of across different IDSs more straight forward, we

  • flatted results so that they are individual items in results[*]

  • simplified samples[*] to better match TetraScience's conventions

  • Added primary keys to samples, systems, and users, and their analogous foreign keys placed in results

Specific changes:

  • Update samples[*].@link to be named samples.pk_samples

  • Update samples[*].ids[*].key/value to be named samples[*].id and samples[*].labels[*]

  • Update results[0].samples[*].results[*].name to be named results[*].key

  • Update results[0].samples[*].results[*].value to be named results[*].value and have the original string value of the sample result

  • Update results[0].samples[*].results[*].unit to be named results[*].numerical_value_unit

  • Update results[0].samples[*].results[*].precision to be named results[*].numerical_value_precision

  • Update results[0] now includes value_data_type, string_value, numerical_value, and boolean_value, fk_samples, fk_systems, and fk_users fields

  • Update results[*].tasks[*] to be named results[*]

  • Add systems[*].pk_systems field

  • Add users[*].pk_users field

  • Add runs[*].name

  • Update users[*].logon_name to users[*].id to follow IDS conventions

  • Update users[*].full_name to users[*].name to follow IDS conventions

  • Update methods[*].id fields:

    • Update methods[*].id.business to methods[*].id

    • Update methods[*].id.internal to methods[*].internal_id

    • Update methods[*].id.method to methods[*].method_id

  • Remove systems[*].software.id

  • Add the following fields under methods[*].metadata.parameters[*]

    • string_value

    • numerical_value

    • boolean_value

    • value_data_type

    • function_name

    • function_property_name

  • Add the following fields under methods[*].metadata.scopes[*].method_parameters[*]

    • string_value

    • numerical_value

    • boolean_value

    • value_data_type

  • Remove the following fields under methods[*].metadata.parameters[*] and methods[*].metadata.scopes[*].method_parameters[*]

    • max_length

    • min_length

    • decimal_places

    • value

Version 1.0.0

Release Date: 6/10/2021

  • Integrate with LabX IntegrationService
  • Output LabX Task as JSON file format
  • Raw files are uploaded to Tetra Data Platform
  • Windows-based UI for easy configuration and processing status monitoring
  • Support both local deployment and remote deployment