Release Date: 10 August 2022 (Updated 23 August 2022)

📘

Quality Report

TetraScience is committed to creating quality software. Software is developed and tested using our TetraScience Quality System. The Quality Report for version 3.3.0 is TPM-50. Customers who have purchased the GxP package can request a copy of the Quality Report. If you have further questions about the Quality Report or the GxP package, please contact your Customer Success Manager.

Upgrade Considerations

During the upgrade, there might be a brief downtime (about 15-30 minutes) when users will not be able to access the platform. This includes the TDP User Interface and the APIs. After the upgrade is complete, customers should check their pipelines for failures or cancellations and reprocess those if found. For more details surrounding the timing of the upgrade, please see your CSM.

❗️

IMPORTANT

Customers still using the API_KEY for authentication must shift to using the JSON Web Token (JWT) before the 3.3.x migration to avoid breaking their integrations. For more information on how to create a JWT, see the https://developers.tetrascience.com/reference/authentication topic.

What's New?

TetraScience has released its next version of the Tetra Data Platform (TDP) version 3.3.0. As its major focus, this release targets greatly enhanced performance increases, admin and compliance effort reduction, and UX and data consistency.

For details on specific feature and functionality improvements, review these TDP version 3.3.0. release notes:

• Enhancements
• Bug fixes
• Deprecated features
• Known issues
• Security

Links to Tetra Agents and Artifacts Release Notes are provided in the Other Release Notes section at the end of this topic.

Enhancements

Enhancements are new features and functionality that have been added to the software.

Performance Increases

• There’s greater throughput of files processed per hour, up to 2.5x out-of-the-box. Contact Customer Support to explore optimizing your environment to increase performance further.
• New Data Pipeline architecture with customizable per environmental parameters to optimize pipeline processing throughput.  This was accomplished with a complete redesign of the scheduling algorithm to fairly distribute workflows. This eliminates the problem of pipelines “starving” one another. Pipeline enhancements were also made to reduce overall costs of processing data.

Administration and Compliance Effort Reduction

• We have a new service that speeds the compliance documentation process for customers who subscribe to our GxP package. Deployments undergo verification and validation testing upon upgrade and on a recurring basis. Automation of this process greatly reduces the effort needed to achieve and maintain GxP compliance. The platform immediately flags any issues, which reduces the window between error occurrence and detection.
• To help customers provide evidence of how the Tetra Data Platform (TDP) is being used for a process governed by GLP (good laboratory practice) guidelines, TetraScience has added change reason as an audit trail option.
• Single sign-on (SSO) functionality is now available on Multi-Tenant environments.  SSO gives our managed customers the ability to integrate with their existing authentication and authorization solutions. (Switching to this will require an additional subdomain in the URL.)
• SSO supports SAML-compatible identity providers (Okta, Azure AD, etc.)
• User password reset has been added as a self-service feature to enhance the user experience while minimizing the tasks a Site Admin would need to perform.
• There are now lower data indexing costs because of the IDS full-text search removal.
• Organization Admins now have access to File Reconciliation pages on Health Monitoring. Please contact TetraScience before using this feature; we will provide hands-on support.

UX and Data Consistency

• The Tetra Data Platform (TDP) UI has been enhanced and redesigned to provide a more intuitive customer experience.
• There’s a single, dockable hamburger menu, so you can see all of the options at a glance and they can remain on screen while you work.
• There are also new icons that remain visible, even if the menu is hidden. This will make it much easier for you to navigate to other pages.
• Your organization is always visible at the bottom of each screen.
• We’ve also made it easier for you to switch organizations, view your account information, and sign out.
• The Pipeline Design screen has been renamed Pipeline Manager. The Pipeline Processing screen is now called File Processing.
• The IoT User Interface in Data Sources has been revamped as well.
• File Info and Attributes (Metadata, Tags, and Labels) are now available via SQL

And More

• Data reconciliation has been expanded to Athena.
• Read Only users now have access to view Agent and Pipeline configurations in the current organization to help with initial setup and any support interactions.
• There are new Pipeline APIs as well.

Bug Fixes

The following are customer-reported bug fixes that were made for this version of the TDP software:

• CON-15 - Fixed the bug where the Cellario connector hung when one event was stuck in the processing state. Events stuck in the processing state now timeout.
• DL-389 - Previously, pipelines that used File - Has Path operator as the trigger condition, there was an inconsistency between files found automatically by a pipeline and files that were found when a user scanned for unprocessed files. This ticket resolved this inconsistency.
• DL-496 - Improved performance surrounding ElasticSearch reconciliation jobs.
• DL-499 - More descriptive error message on the File Details page when an invalid File is pulled up by directly entering a URL.
• DL-507 - Maintained text box focus was automatically removed from the text input box after each letter typed.
• DL-526 - Made changes to the UI to more visibly display the Add Label button for files with more than 10 existing labels.
• DP-32 - Text search is no longer case sensitive on the pipeline processing page.
• DP-38 - Cache issue was fixed; workflows no longer fail due to throttling rate.
• DP-39 - ThrottlingException errors now appear in the logs.
• DP-45 - The namespace for master/task scripts displayed on the Pipeline Design Page now matches the selected script's namespace.
• DP-46 - Workflow status now correctly displays.
• DP-49 - Scalability now mitigates many load-related issues.
• DP-63 - If the protocol is updated, the first workflow no longer fails automatically.
• DP-66 - Protocol changes are now saved properly.
• DP-74 - Timeout fixes have been made to ensure that pipelines are not stuck in the “in progress” state.
• DP-105 - Checkboxes now accurately display which files need to be processed.
• DP-275 - MassLynx task no longer fails after syncing.
• DP-299 - Pipeline scaling performance has been improved.
• DP-352 - <label name> <exists> has been removed from the list of pipeline trigger conditions.
• PLCR-291 - The Entity name from the local Agent is now recorded in the audit trail log.
• PLCR-425 - Label updates by service users using all POST methods are now logged in the audit trail. Previously, users had to use --data-raw as an option to trigger the audit trail in this scenario.
• PUI-386 - Screens where attributes are added and edited now look and behave more consistently.

Deprecated Features

These are the features that have been deprecated in the product:

• The Athena metadata_v1 table has been deprecated. Please transition queries and dashboards to use file_info_v1 which has the same fields and more attributes about all file categories.
• To improve performance, full-text search (FTS) for IDS files has been deprecated and we plan to deprecate the functionality for RAW data in a future release.
  • For customer-hosted environments:
    • Deprecating FTS for IDS files provides additional computational and storage cost savings.
    • To ensure particular FTS search use cases are still achievable, TetraScience has the ability to turn this on via an AWS parameter or to turn off FTS indexing for RAW data to further save indexing costs. Please contact support if you wish to do this.
• The Apps and Email menu options have been removed from TDP. Existing links to Streamlit dashboards will continue to work with no functionality updates and are supported through TetraScience professional services team.
• Pipeline reserved instances have been removed. A new selection algorithm manages resources to make sure files are processed by pipelines in a timely manner and at scale.
• Pipeline Dashboard has been removed from the main TDP Navigation. It is temporarily accessible via URL (https:<tdp-env>.com/dashboard), but will be removed in a future release. Content has been integrated into the new Pipeline File Processing page.

Known Issues

These are the known issue(s) for this release of the TDP software:

• AG-1980 - On RHEL8 datahub instances, AWS CLI, version 1 is incorrectly identified as AWS CLI version 2, which causes errors. WORKAROUND: Use AWS CLI, version 2 instead.
• AG-2041 - The L7 Proxy does not explicitly support custom private networks. If customers have agents installed on machines with different subnets, the machines might fail to connect to datahub when S3 Direct Upload is enabled. WORKAROUND: A workaround is available; please contact Customer Success for more details.
• DL-212 - The file details page does not show a workflow history unless you specify a pipeline ID. WORKAROUND: To see the workflow history, go to the workflow processing page instead.
• DL-217/DL-218/DP-116 - Files that have special characters (( { [ ) appear to be in the wrong order and are split in the file path when you browse in the Search window. However, this is just a visual issue; if you search for the path, the files are returned correctly.
• DL-219 - Users can't edit Egnyte/Box sources if their folder tree paths consist of special characters. WORKAROUND: Avoid the use of those special characters in folder names.
• DL-708 - Adding or removing metadata/tags via API removes all labels applied on the file unless an additional label API is used. WORKAROUND: Use the additional label API.
• IOT-244 - For Change Reason enabled organizations, when updating an IoT Agent with attached labels, the change Reason entry will be triggered twice. WORKAROUND: Enter the change reason twice.
• IOT-246 - Agent Name can only be configured for up to 256 characters, otherwise, Agent will appear offline. WORKAROUND - Names should be less than 256 characters.

Miscellaneous

There were several Dev Ops changes related to the IoT Layer:

• Updated IoT Layer (AnyLink Cloud): v4.1.x to v4.2.14.
  • The Network load balancer was introduced in v4.2 so that the port for the web UI no longer needs to be appended to the URL to get to the IoT Layer. (Example: https://iot.some-domain.com:8189 is now just https://iot.some-domain.com).
  • Other external ports remain the same and are accessed through the load balancer now.
  • The network load balancer is integrated with the AWS certificate manager
  • Support for custom resource tags for the EC2 instance.
• There are no audit trails recorded for agent configuration actions (create, update, enable, disable) because new APIs have been implemented.
• Postgres DB updates in the backend caused the following changes:
  • Existing data being stored in anylink.public has been migrated to anylink.anylink_schema
  • If troubleshooting or monitoring db, look for db: anylink and schema: anylink_schema
  • Once upgraded, agents and devices should remain and be visible on TDP UI however, some of the new buttons such as the “Configuration File <>" button that displays the current moduleConfig.xml will not display anything. You will need to re-set up the agent to see this.

Security

TetraScience continually assesses and addresses security-related enhancements and bugs. Various vulnerabilities, as well as NodeJS upgrades have been addressed.

Other Release Notes

Release notes for Artifacts and Agents are here.