The following deprecation notices are for TetraScience products including the Tetra Data Platform, Tetra Agents, and more. For all deprecation notices prior to those listed on this page, see the release notes located here: Tetra Data Platform (TDP) Release Notes.

Deprecations are components, features, or functions that are no longer supported or maintained by TetraScience. For more information about deprecations, see this page: Tetra Product Deprecation Process Overview.

The following table summarizes the deprecations. To learn more about each deprecation, click the corresponding link in the Details column.

Deprecation Notices

The following are deprecation notices for Tetra Products.

Metadata and Tags Deprecation

Starting in TDP version 3.6.0, the metadata and tags attributes are on a deprecation path. Functionality for labels will be improved to further meet FAIR (findable, accessible, interoperable, and reusable) data principles and incorporate critical parts of existing metadata and tags functionality.

🚧

IMPORTANT

No breaking changes will occur with the metadata and tags deprecation until a customer switches to using labels only. TetraScience will provide customers with a simple migration process to transfer their existing pipelines and searches to using labels. Please contact your customer success manager (CSM) for more information.

The metadata and tags deprecation provides the following benefits:

• Simplifying the process of adding attributes to files
• Increasing general compliance with FAIR data principles
• Reducing human, processing, and data costs associated with attribute management
• Reducing the need for file reprocessing
• Increasing scalability and automation
• Providing more precise search results

Starting Immediately (For TDP v3.5.x)

Customers should begin using labels as the only attribute to store each file’s contextual information. When metadata and tags are fully deprecated, labels will be the only attribute available in the TDP.

For TDP v3.6.x

TetraScience plans to implement the following changes in TDP v3.6.x:

• Reducing the prominence of metadata and tags in the TDP user interface

For TDP v3.7.x

TetraScience plans to implement the following changes in TDP v3.7.x:

• Updating labels to maintain version history for better auditing and transparency, and to maintain the association of labels to versions of underlying files
• Helping to copy all existing metadata and tags to labels, so that all contexts of the primary data is searchable with labels (This migration will happen as part of customers' upgrade to TDP v3.7.0)
  • Keep in mind that the context that is represented by metadata and tags may start to be out of sync with the consolidated context represented by labels.
  • Any exceptions will be automatically handled by the TDP during this migration.
  • Going forward, any metadata and tags that are created will also be created as labels in a dual-write approach
• Providing a pathway for customers to self-migrate existing metadata and tags to labels.
  • Labels will accommodate metadata and tags by using the following rules:
    • Metadata {"location": "Chicago"} will become label [{"name": "location", "value": "Chicago"}]
    • Tag ["validated", "decorated"] will become label [{"name": "tag", "value": "validated"}, {"name": "tag", "value": "decorated"}]
    • Any exceptions (namespace violations, character violations) will be identified for self-remediation.

📘

NOTE

No metadata or tags will be deleted.

For Future Versions of the TDP

TetraScience plans to implement the following changes in future versions of the TDP:

• Allowing metadata and tag context to be viewed through the TDP user interface, but no new metadata or tags to be created
• Preventing TDP user interface and TetraScience API users from creating or using metadata or tags.
  • Users will need to use labels for new configurations for agents, data sources, and pipeline triggers.
  • Existing pipelines and data sources will continue to function with existing metadata and tags.
  • Underlying data for metadata and tags won’t be removed.
  • EQL and SQL queries referencing metadata and tags will continue to work.
• Preventing TDP users from updating or creating new metadata and tags for files.
  • Users will still be able to add metadata and tags to specific files by using the TetraScience API to maintain customers' existing programmatic calls.
  • Metadata and tags will be hidden in the TDP user interface and TetraScience API.
  • Metadata and tags will remain as data for historical reference.

At the End of the Deprecation Process

• All existing metadata and tags data will be copied as labels and maintained as historical data.
• Metadata and tags will no longer be a concept for setup and management of data in the TDP and Tetra Agents.
• Existing pipelines and TetraScience API updates to files for metadata and tags set up prior to TDP v3.6.0 will continue to work. Any new metadata and tags will be written as labels.
• Programmatic search queries (EQL and SQL) that reference metadata and tags will continue to work, but will only return data from deprecated pipelines or data sources that were set up prior to TDP v3.6.0.

ES and FileInfo API Deprecation - Integration and Source.Integration

Cloud Data Connector Deprecation

We are deprecating the concept of Cloud Data Connector (CDC) to simplify agent configuration. As of 3.4, creating CDCs and adding agents to existing CDCs will be disabled. If your agent needs a Datahub and Generic Data Connector (GDC), agent creation will be unchanged. If you don’t, new Agents can be created from the Agents page with ‘No Connector’ selected.

Existing CDCs will be unaffected for 3.4, but is scheduled to be removed in the 3.5 release. All Agents without a Datahub V1 + GDC will display ‘No Connector’ in the user interface once the CDCs are removed.

DataHub V1 Deprecation

An improved design of the DataHub, “Tetra Hub“ is under development and internal testing. Starting from 3.6 of the software (Q2 2023 release), you will not be able to create a new DataHub, however, existing DataHubs will continue to function. Starting from 3.6 of the TDP, you will only be able to create “Tetra Hub“ (you can consider this an improved version of DataHub)

• Tetra Hub requires Linux OS for its host machine, is expected to support Ubuntu OS only and may have unofficial support of RedHat OS.
• Tetra Hub requires network access from the Hub host machine to the TDP API. Hub supports customers to define their own proxies in between.

Athena metadata_v1 Table Deprecation

For version 3.3.0 of TDP, the Athena metadata_v1 table has been deprecated. Please transition queries and dashboards to use file_info_v1, which has the same fields and more attributes about all file categories. Tetrascience does not plan to release new versions of this table and starting from v3.6 (~June 2023) will remove this table from the TDP.

Full-Text Search Deprecation

For version 3.3.0 of the TDP, to improve performance, full-text search (FTS) for IDS files has been deprecated and we plan to deprecate the functionality for RAW data in a future release.

For customer-hosted environments:

• Deprecating FTS for IDS files provides additional computational and storage cost savings.
• To ensure particular FTS search use cases are still achievable, TetraScience has the ability to turn this on via an AWS parameter or to turn off FTS indexing for RAW data to further save indexing costs. Please contact support if you wish to do this.

Apps and Email Menu Options Removed from TDP

For version 3.3.0 of the TDP, the Apps and Email menu options have been removed from TDP. Existing links to Streamlit dashboards will continue to work with no functionality updates and are supported through TetraScience professional services team.

Pipeline Reserved Instances Deprecation

For version 3.3.0 of the TDP, the pipeline reserved instances have been removed. A new selection algorithm manages resources to make sure files are processed by pipelines in a timely manner and at scale.

Pipeline Dashboard Deprecation

For version 3.3.0 of the TDP, the Pipeline Dashboard has been removed from the main TDP Navigation. It is temporarily accessible via URL (https:<tdp-env>.com/dashboard), but will be removed in a future release. Its content has been integrated into the new Pipeline File Processing page.