API DocumentationProduct Documentation
Product Documentation

TDP v3.6.3 Release Notes

Suggest Edits

Release date: 1 February 2024 (Last updated: 7 March 2024)

TetraScience has released its next version of the Tetra Data Platform (TDP), version 3.6.3. This release focuses on implementing a security update for Tetra Hub and introduces several user interface improvements for monitoring events.

Here are the details for what’s new in TDP v3.6.3.

📘

Security

TetraScience continually monitors and tests the TDP codebase to identify potential security issues. Various security updates are applied to the following areas on an ongoing basis:

  • Operating systems
  • Third-party libraries

📘

Quality Management

TetraScience is committed to creating quality software. Software is developed and tested following the ISO 9001-certified TetraScience Quality Management system. This system ensures the quality and reliability of TetraScience software while maintaining data integrity and confidentiality.

New Functionality

New functionalities are features that weren’t previously available in the TDP.

  • There is no new functionality in this release.

📘

GxP Impact Assessment

All new TDP functionalities go through a GxP impact assessment to determine validation needs for GxP installations. New Functionality items marked with an asterisk (*) address usability, supportability, or infrastructure issues, and do not affect Intended Use for validation purposes, per this assessment. Enhancements and Bug Fixes do not generally affect Intended Use for validation purposes, and items marked as Beta Release are not suitable for GxP use.

Enhancements

Enhancements are modifications to existing functionality that improve performance or usability, but don't alter the function or intended use of the system. The following are new enhancements introduced in TDP v3.6.3.

Enhancements for Data Integrations

User Interface Improvements for Event Monitoring

To help simplify and streamline the TDP user interface, the the following UI elements are now renamed:

  • The Events Timeline tab on the File Details page is now File Journey.
  • The Integration Events tab on the Health Monitoring page is now Events.

For more information, see View the File Details Page and Monitor Events.

Agents Set Up without a Tetra Hub Now Have Offline Alarms

Tetra Agents configured with a TDP (No Connector) connection now have an Offline Alarm. The offline alarm indicates when an Agent is offline by using the Heartbeat Metric, which validates the connection (heartbeat) between the TDP and the Agent every minute. These infrastructure-level alerts contain no sensitive information. They indicate which Agent failed and how only, and are sent to TetraScience by AWS automatically. Customers can also configure Offline Alarm emails to be sent to a specific email address when they configure an Agent.

For more information, see Logs, Metrics, and Alerts.

Bug Fixes

The following bugs have been fixed.

Data Access and Management Bug Fixes

  • An issue found during internal testing where a subset of datacube data wasn't available in Amazon Athena is now resolved. The impact is limited to data that meets the following criteria only:

    • The data is ingested by Tetra Empower Agent v5.1 and higher
    • The data is generated by the Empower Tetra Data IDS (lcuv-empower - v14.0.0 and v15.0.0)
    • The channel type is 2D or 2D-derived
    • The channel description doesn't have a wavelength

    After updating to TDP v3.6.3, if customers require access to datacube data that meets these criteria through SQL queries, the affected Empower data must be reprocessed. For more information, see Reprocess Files. If you have further questions, please contact your customer success manager (CSM) for support.

Deprecated Features

There are no new deprecated features in this release. For more information about TDP deprecations, see Tetra Product Deprecation Notices.

Known and Possible Issues

The following are known and possible issues for TDP v3.6.3.

Data Integrations Known Issues

  • In rare configurations, data ingested by Tetra Empower Agent v5.1.0 and higher isn't indexed for SQL access. This defect only impacts indexing for data that meets the following criteria:

    • The data is ingested by Tetra Empower Agent v5.1 and higher
    • The data is generated by the Empower Tetra Data IDS (lcuv-empower - v14.0.0 and v15.0.0)
    • The data has a wavelength measure {scale} value that’s set to empty.

    A fix for this issue is included in TDP v3.6.4. For more information, see the TDP v3.6.4 Release Notes.

  • On the Command Details page, If a command has no response (for example, if the request's status is Pending), the Response section displays the following error:

     "ERROR":{1 item
 "message":"src property must be a valid json object"
 }

    When this error appears, command processing isn't affected and no action is needed. A fix for this issue is in development and testing and is scheduled for a future TDP release. For more information, see View Command Details.

  • Pluggable Connector offline alarms aren't created when a Connector is created. To create these offline alarms, customers must select Sync on the Connector Details page after the Connector is online. A fix for this issue is in development and testing and is scheduled for TDP v3.7.0. For more information, see Change a Pluggable Connector’s Status.

  • When installing a Tetra Hub on a host server that already has an AWS Systems Manager registration key, the Amazon ECS container agent startup fails. An AccessDenied error is then logged in the agent’s Amazon CloudWatch Logs. In TDP v3.6.0, the Hub installer automatically detects the issue and provides instructions to fix it.

  • When installing or rebooting a Tetra Hub, the Hub’s Health status incorrectly displays as CRITICAL for a short time in the TDP UI. After the TDP receives the Hub’s initial metrics and proxy status, the Hub’s status displays as Online. No action is needed, and no alarms or notifications are generated.

  • The Events tab on the Health Monitoring Dashboard might present a spinner if an Agent is configured with no file path (filePath) and hasn't produced any file events (fileEvents).

Data Harmonization and Engineering Known Issues

  • In Browse view on the Search Files page, the Edit Labels on <#> Searched Files action processes all of an organization’s files in the Data Lake, not just the searched files. A fix for this issue is in development and testing and planned for TDP v4.0.0. List view on the Search Files page is unaffected by this defect.
  • Files with more than 20 associated documents (high-lineage files) do not have their lineage indexed by default. To identify and re-lineage-index any high-lineage files, customers must contact their CSM to run a separate reconciliation job that overrides the default lineage indexing limit.
  • Elasticsearch index mapping conflicts can occur when a client or private namespace creates a backwards-incompatible data type change. For example: If doc.myField is a string in the common IDS and an object in the non-common IDS, then it will cause an index mapping conflict, because the common and non-common namespace documents are sharing an index. When these mapping conflicts occur, the files aren’t searchable through the TDP UI or API endpoints. As a workaround, customers can either create distinct, non-overlapping version numbers for their non-common IDSs or update the names of those IDSs.
  • File reprocessing jobs can sometimes show less scanned items than expected when either a health check or out-of-memory (OOM) error occurs, but not indicate any errors in the UI. These errors are still logged in Amazon CloudWatch Logs. A fix for this issue is in development and testing.
  • File reprocessing jobs can sometimes incorrectly show that a job finished with failures when the job actually retried those failures and then successfully reprocessed them. A fix for this issue is in development and testing.
  • On the Pipeline Manager page, pipeline trigger conditions that customers set with a text option must match all of the characters that are entered in the text field. This includes trailing spaces, if there are any.
  • File edit and update operations are not supported on metadata and label names (keys) that include special characters. Metadata, tag, and label values can include special characters, but it’s recommended that customers use the approved special characters only. For more information, see Attributes.
  • The File Details page sometimes displays an Unknown status for workflows that are either in a Pending or Running status. Output files that are generated by intermediate files within a task script sometimes show an Unknown status, too.

Data Access and Management Known Issues

  • File events aren’t created for temporary (TMP) files, so they’re not searchable. This behavior can also result in an Unknown state for Workflow and Pipeline views on the File Details page.
  • When customers search for labels that include @ symbols in the TDP UI’s search bar, not all results are always returned.
  • When customers search for some unicode character combinations in the TDP UI’s Search bar, not all results are always returned.
  • If customers modify an existing collection of search queries by adding a new filter condition from one of the Options modals (Basic, Attributes, Data (IDS) Filters, or RAW EQL), but they don't select the Apply button, the previous, existing query is deleted. To modify the filters for an existing collection, customers must select the Apply button in the Options modal before you update the collection. For more information, see How to Save Collections and Shortcuts.

TDP System Administration Known Issues

  • The latest Connector versions incorrectly log the following errors in Amazon CloudWatch Logs:
    • Error loading organization certificates. Initialization will continue, but untrusted SSL connections will fail.
    • Client is not initialized - certificate array will be empty
      These organization certificate errors have no impact and shouldn’t be logged as errors. A fix for this issue is currently in development and testing, and is scheduled for an upcoming release. There is no workaround to prevent Connectors from producing these log messages. To filter out these errors when viewing logs, customers can apply the following CloudWatch Logs Insights query filters when querying log groups. (Issue #2818)

CloudWatch Logs Insights Query Example for Filtering Organization Certificate Errors

fields @timestamp, @message, @logStream, @log
| filter message != 'Error loading organization certificates. Initialization will continue, but untrusted SSL connections will fail.'
| filter message != 'Client is not initialized - certificate array will be empty'
| sort @timestamp desc
| limit 20
  • If a reconciliation job, bulk edit of labels job, or bulk pipeline processing job is canceled, then the job’s ToDo, Failed, and Completed counts can sometimes display incorrectly.

Upgrade Considerations

During the upgrade, there might be a brief downtime when users won't be able to access the TDP user interface and APIs.

After the upgrade, the TetraScience team verifies that the platform infrastructure is working as expected through a combination of manual and automated tests. If any failures are detected, the issues are immediately addressed, or the release can be rolled back. Customers can also verify that TDP search functionality continues to return expected results, and that their workflows continue to run as expected.

For more information about the release schedule, including the GxP release schedule and timelines, see the Product Release Schedule.

For more details about the timing of the upgrade, customers should contact their CSM.

Other Release Notes

To view other TDP release notes, see Tetra Data Platform Release Notes.

Updated 8 months ago