(Update: 27 Feb 2023)
The following deprecation notices are for TetraScience products including the Tetra Data Platform, Tetra Agents, and more. For all deprecation notices prior to those listed on this page, see the release notes located here: Tetra Data Platform (TDP) Release Notes.

Deprecations are components, features, or functions that are no longer supported or maintained by TetraScience. For more information about deprecations, see this page: Tetra Product Deprecation Process Overview.

The following table summarizes the deprecations. To learn more about each deprecation, click the corresponding link in the Details column.

Deprecation Notices

The following are deprecation notices for Tetra Products.

Metadata and Tag Deprecation

In TDP, there are three entities that can be used to contextualize or describe the primary data – Metadata, Tags, and Labels.

Starting in TDP v3.6.0, Tetrascience will start to deprecate and ultimately remove feature support for Metadata and Tags as a concept for TDP. Functionality for labels will be improved to further meet FAIR (Findable, Accessible, Interoperable and Reusable) needs and incorporate critical parts of existing Metadata and Tags functionality.

The deprecation of Metadata and Tags will simplify the process of adding attributes to customer files. Using Labels instead of Metadata and Tags will increase general FAIR compliance, reduce the need for file reprocessing, increase scalability and automation, and allow customers to obtain more precise search results. This change will make adding attributes a more seamless, intuitive process resulting in a much better user experience. It will also reduce human, processing and data costs associated with attribute management and will reduce the upfront effort needed to configure attributes within the platform.

Starting immediately, Customers should begin using only Labels when the choice between Metadata, Tags, and Labels is present. When Metadata and Tags are fully deprecated, Labels will be the sole entity available for associating attributes with files.

In 3.6 we plan to:

• Update Labels to maintain version history for increased auditability and transparency, and maintain the association of Labels to versions of underlying files.
• Reduce the prominence of Metadata and Tags on UI configuration and view screens in TDP while promoting Labels.
• Provide a pathway for users of TDP to self-migrate existing Metadata and Tags to Labels.
  • Labels will accommodate Metadata and Tags following the rules below.
    • Metadata {"location": "Chicago"} will become Label [{"name": "location", "value": "Chicago"}]
    • Tag ["validated", "decorated"] will become Label [{"name": "tag", "value": "validated"}, {"name": "tag", "value": "decorated"}]
    • Any exceptions (namespace violations, character violations) will be identified for self-remediation.
  • Note: No Metadata or Tags will be deleted.

In 3.7 we plan to:

• Help copy all existing Metadata and Tags to Labels, so that all contexts of the primary data is searchable with Labels. Note that this will happen as part of the upgrade to TDP 3.6.
  • This means that the context that is represented by Metadata and Tags may start to be out of sync with the consolidated context represented by Labels.
  • Any exceptions will be automatically handled by the platform during this migration.
  • Going forward, any created Metadata and Tags will also be created as Labels in a dual write approach.
• Allow Metadata and Tag context to be viewed, but no new Metadata and Tags will be able to be created through the UI.
• Prevent UI and API users from setting up, creating, editing pipelines, data sources using Metadata and Tags. Users will be forced to use labels for new configurations for agents, data sources, and pipeline triggers.
  • Existing pipelines, data sources will continue to function with Metadata and Tags.
  • There will be no removal of underlying data for Metadata and Tags.
  • EQL and SQL queries referencing Metadata and Tags will continue to function.
• Prevent UI users from updating or creating new Metadata and Tags for files.
  • Users will still be able to add Metadata and Tags to specific files using the API to maintain customers' existing programmatic calls.

In 3.8+ we plan to:

• Hide Metadata and Tags in the UI and API. Metadata and Tags will remain as data for historical reference. This will not be done until all historical Metadata and Tag data and file associations have been copied as Labels.

At the end of this deprecation process:

• All existing Metadata and Tags data will be copied as Labels, and maintained as historical data.
• Metadata and Tags will no longer be a concept for setup and management of data in TDP and Agents.
• Existing Pipelines and API updates to files for Metadata and Tags set up prior to TDP 3.6 will continue to function and any new Metadata and Tags will be written as Labels.
• Programmatic search queries (EQL and SQL) which reference Metadata and Tags will still function but will only return data from deprecated pipelines or data sources set up prior to 3.6 for Metadata and Tags.

ES and FileInfo API Deprecation - Integration and Source.Integration

Cloud Data Connector Deprecation

We are deprecating the concept of Cloud Data Connector (CDC) to simplify agent configuration. As of 3.4, creating CDCs and adding agents to existing CDCs will be disabled. If your agent needs a Datahub and Generic Data Connector (GDC), agent creation will be unchanged. If you don’t, new Agents can be created from the Agents page with ‘No Connector’ selected.

Existing CDCs will be unaffected for 3.4, but is scheduled to be removed in the 3.5 release. All Agents without a Datahub V1 + GDC will display ‘No Connector’ in the user interface once the CDCs are removed.

DataHub V1 Deprecation

An improved design of the DataHub, “Tetra Hub“ is under development and internal testing. Starting from 3.6 of the software (Q2 2023 release), you will not be able to create a new DataHub, however, existing DataHubs will continue to function. Starting from 3.6 of the TDP, you will only be able to create “Tetra Hub“ (you can consider this an improved version of DataHub)

• Tetra Hub requires Linux OS for its host machine, is expected to support Ubuntu OS only and may have unofficial support of RedHat OS.
• Tetra Hub requires network access from the Hub host machine to the TDP API. Hub supports customers to define their own proxies in between.

Athena metadata_v1 Table Deprecation

For version 3.3.0 of TDP, the Athena metadata_v1 table has been deprecated. Please transition queries and dashboards to use file_info_v1, which has the same fields and more attributes about all file categories. Tetrascience does not plan to release new versions of this table and starting from v3.6 (~June 2023) will remove this table from the TDP.

Full-Text Search Deprecation

For version 3.3.0 of the TDP, to improve performance, full-text search (FTS) for IDS files has been deprecated and we plan to deprecate the functionality for RAW data in a future release.

For customer-hosted environments:

• Deprecating FTS for IDS files provides additional computational and storage cost savings.
• To ensure particular FTS search use cases are still achievable, TetraScience has the ability to turn this on via an AWS parameter or to turn off FTS indexing for RAW data to further save indexing costs. Please contact support if you wish to do this.

Apps and Email Menu Options Removed from TDP

For version 3.3.0 of the TDP, the Apps and Email menu options have been removed from TDP. Existing links to Streamlit dashboards will continue to work with no functionality updates and are supported through TetraScience professional services team.

Pipeline Reserved Instances Deprecation

For version 3.3.0 of the TDP, the pipeline reserved instances have been removed. A new selection algorithm manages resources to make sure files are processed by pipelines in a timely manner and at scale.

Pipeline Dashboard Deprecation

For version 3.3.0 of the TDP, the Pipeline Dashboard has been removed from the main TDP Navigation. It is temporarily accessible via URL (https:<tdp-env>.com/dashboard), but will be removed in a future release. Its content has been integrated into the new Pipeline File Processing page.