TDP v3.1.0 Release Notes
Release Date: 14 July 2021
This topic provides information on enhancements, bug fixes, deprecated features, as well as minor functionality changes for version 3.1.0 of the Tetra Data Platform (TDP). Links to the release notes for Agents and the SDK appear in the other release notes that appear near the bottom of these release notes.
Enhancements (New Features and Functionality)
There were several notable enhancements in this version of the software.
Search Improvements
The search functionality has been enhanced with features to make it easier to find the information you are looking for.
- File Path Hierarchies, which are based on the customer's file paths, are now displayed.
- Full-Text search is now available and allows you to search for any string of text in a document. Boolean operators such as "AND" & "OR" are also supported. Full-Text Search is available for RAW and IDS documents. Currently supported RAW object types: JSON, XML, RTF, and HTML.
- Four new filter panels have been added to make more complex searches easier: Basic Searches, Attributes, Data (IDS) Filters, Raw EQL. You can now easily search for custom or platform metadata, and view project data that is usually indexed as part of the JSON object (e.g. project ID).
- You can also run and test your EQL from the Tetra Data Platform.
- Related input and output files are now displayed in the file info section.
NOTE:
Note that full-text search is only available for documents ingested after upgrading to V3.1.0. Starting with the next release, full-text search will be available for all documents, regardless of when they were ingested.
Cloud Configuration
Consolidates the creation of agents in the Tetra Data Platform so that the administrator can manage all agents in one place. In past versions of the software, you would have needed to create an agent in either the Data Hub Data Management or the User Defined Integration screens. Now, on a single screen, the administrator can view additional details about a specific agent, such as whether an agent connected to the Tetra Data Platform is online or offline or if the queue is enabled.
For this version of the software, you can also configure the File-Log Agent from the Cloud Configuration screen. Note that the User Defined Integration has been renamed as Cloud Data Connector for simplicity and consistency.
IDBS E-Workbook Connector
The Tetra IDBS E-Workbook Connector has been added to the TDP. This connector allows you to connect IDBS Version 9.x desktop on-premises and IDBS Version 10.x desktop and cloud versions of the e-workbook to TDP.
Health Monitoring
Health Monitoring dashboards provide an easy way for Tetra Data Platform Organizational Administrators to assess the health and performance of Tetra Data Platform components. There are six different dashboards that you can use to monitor the health of the different components: An Overall dashboard (simply named "Dashboard") that gives you a snapshot of the health of the entire Tetra Data Platform ecosystem, Agent, Data Hub, Connector, Pipeline, and File.
Token Enhancements
Token expiration times can now either never expire or can be valid for a total of 720 days. Note that tokens can remain active for much shorter time periods as well. Additionally, service users are now limited to one token per user at a time. Note that service users that have multiple tokens and that were created prior to version 3.1 of the software are not affected.
Design Refresh
The user interface colors have been changed to reflect the new TetraScience branding.
Documentation Updates
Documentation has been updated and reorganized to make finding information easier. New documentation links appear above.
Bug Fixes
The following bug fixes were made to the platform.
-
DI-4580 – Update Users Name (SSO)
If a user’s name is updated in their Active Directory account the user login now uses the same name if the email address remains the same. -
DI-4358 - Unable to Process Unprocessed Files (Internal HTTP 504 Gateway Timeout)
Modified and optimized the code to ensure that large volumes of files can be reprocessed if needed. -
DI-4353/DI-4354 - Modify GraphQL to Return Correct Versions of Files Based on the Selected workflow/Add “Search by Workflow ID” to FileInfo API
Modified code to ensure that the correct versions of files can be returned based on the selected workflow. -
DI-4343 - Egnyte Connector Slow
Disabled the member role user's ability to delete a file on the pipeline processing page. -
DI-4314 - Selected Files in Pipeline Processing Page
Resolved the UI issue that did not correctly report the number of selected pipeline files. The correct number now currently appears on the pipeline processing screen. -
DI-4273 - Connector Issue Capturing [File]
Resolved this issue by increasing the interval needed to capture files of this type. -
DI-4263 - Empower Agent Version if Not Updated In UI
The correct version number now appears after an upgrade. -
DI-4172 – Custom Metadata Keys Not Shown When Adding Attributes on v3.0
Added a tip that explains that removing the metadata property in the Attribute Management screen does not remove the value from the files. -
DI-3805 – No Logs Found for Different Workflows
Code fixed to make retrieval of log data occur even if a Windows task fails. Logs are now kept indefinitely. -
DI-3739 – Incorrect Usage of Shared Settings in TDP UI
The Usages column now includes the current usage of each shared setting used in a pipeline parameter.
Deprecated Features
The following features are currently deprecated.
-
Usage of the API Key for Authentication
The API Key, which is used for authentication, is deprecated. While the API Key still works, we strongly recommend that you use the JSON Web Token (JWT) instead. -
Pipeline Dashboard
While the pipeline dashboard is still available, its functionality has been relocated to the Pipeline Monitoring and Pipeline Processing screens.
Minor Functionality Changes
The following minor functionality changes have been made to the product.
-
Status Indicators for Agent File Paths
Before this version of the software, the agent would validate each file path before the agent is started (is placed online). If any of the file paths could not be reached or verified, file collection for all file paths was blocked. Now, when the agent is started, each file path is verified to ensure that it can be reached. If a file path is verified, the agent starts to collect information from files in those paths; if the file path cannot be reached (e.g. a server is offline) it does not block the collection of file paths that have been verified. Visual indicators show whether a file path could be reached. (Done) -
Pipeline UI Changes
The retry functionality and UI duration information has been relocated to the Workflow detail pages. -
Shared Settings and Service User Feature Relocation
Shared Settings (which includes Shared Secrets and Shared Settings) as well as Service User features have been removed from the Organization Page and are now in their own separate pages. -
URL Redirect
In the previous version of TDP, if you were trying to load a TDP screen, but your log in session expired, you were prompted to log in again. The first screen you were shown was the Search screen. Now, if you are trying to load a TDP screen, but your log in session has expired, you will be prompted to log in again, but the first screen you will be shown will be the one you were looking for. For example, if you try to view the Pipeline Design page, but your log in session has expired, you will be prompted to log in again. Upon successful log in, you will see the Pipeline Design page.
Security Updates
TetraScience has upgraded https-proxy-agent, which is vulnerable to a man-in-the-middle attack. TetraScience has fixed this vulnerability with TDP 3.1.0. We strongly suggest that customers upgrade to TDP 3.1.0. For more information see GHSA-pc5p-h8pf-mvwp.
Known Issue(s)
The following known issue(s) for this release appear below.
- DI-4744 Users can save IDBS source with an invalid Date/time of most recent activity
When you set up the IDBS connector, the Query activities from date/time will accept an invalid date if you enter it manually. The workaround is to use the calendar (which you will see when you click on the field), to enter the date instead of entering it manually.
Other Release Notes
Other release notes for different TDP components appear in the following places: