IT Administrators can now fix existing data problems (from previous errors or failures) and prevent new ones using the new Files Reprocessing feature from the Health Monitoring page. This helps to ensure data integrity and consistency across storage including Amazon S3, Elasticsearch (ES), and the TDP FileInfo Service. IT Admins can explore the discrepancies and fix each one of them individually, or in bulk.

The overall TDP Health Monitoring Dashboard has been enhanced to show an improved end-to-end snapshot of the components' health for the entire TDP ecosystem. A new Latest Status column has been added (located between the Name and Health Description columns) to the Dashboard tab on the Health Monitoring page. This column contains the View History link for each item.

• These file details (namespace, slug, and version) have been added to the search results.
• The IDS option from the Platform Metadata File Category on the Search File > Options > Attributes tab was changed to IDS Type to reflect a more accurate filter descriptor.
• Removed has been renamed to Removed Sources, and a new red trash icon (indicating Removed Sources) has been added to the Browse view on the Search page.
• The Elasticsearch preview has been revised by removing these unnecessary UI items for RAW files and PROCESSED files: labels.nameError, labels.valueError, labels.warning, labels.labelId, outputFiles, and inputFiles.
• Navigation within the Search feature has been improved:
• To avoid unnecessary scrolling, the ability to expand and collapse Source Types and Pipelines was added to the side panel of the Search page. Additionally, you can now search within the filter or facet values. For example, you can click Source Type and then start typing “Empower”.
• You can now resize the Folder names column on the Search page in Browse mode to view folder names that appear truncated.
• In addition to clicking the ^ to close the Options screen, you can now click anywhere outside of the screen to close it and retain any settings.
• The Manual Upload folder has been renamed to Manual Uploads and re-positioned to display at the top of the list of folders in Browse mode. (Was released in v3.1.6 as DI-5877)
• You can now click a Preview icon to preview an uploaded file for these added valid files and file types:
  • Images with file type: .png, .jpeg, .gif, or .bmp (including 360-degree images)
  • .pdf, .csv, .xlsx, and .docx file types
  • Video with file type: .mp4 or .webm
  • Audio with file type .mp3
• If any file (included in the valid file types) is greater than 50 MB, then a warning message displays indicating that the file size is too large to display in a preview.

• ASCII printable characters (decimal codes from 32 to 126, inclusive)
• Newline characters (CR/LF...decimal codes of 10 and 13)

DI-6132: When uploading a new file version with a new label, the previous file version's label remained and was added to the new file version resulting with two different labels.
Workaround: First, upload the file without modifying any attributes. Then, after the new version of the file has been uploaded, add the new label. For details on how to add a label using the UI see the Applying Metadata, Tags, and Labels topic

DI-6140: When uploading a new file version, the new version re-writes file's extension using the previously uploaded file.
Workaround: The new file version must have the same name as the previous file.

DI-6805: Pipeline trigger and scan conditions are inconsistent.
Workaround: When creating a pipeline trigger to search for part of a file path, you should use contains instead of has path or is.
This is because the pipeline’s trigger condition treats has path or is differently:

• The trigger condition requires a full match; whereas the scan condition will find partial paths due to the current implementation of how it retrieves results from Elasticsearch (ES). For example (using this file: /user1/fileabc/test.json), the inconsistent behavior occurs when you enter has path or is for a partial match.
• If you set the trigger condition to File Path has path test.json or File Path is test.json, and then upload the file, it will not trigger the pipeline (which is the correct behavior). However, if you scan for unprocessed files for this pipeline, then it will find the file (which is the inconsistent behavior).
• If you want /user1/fileabc/test.json to trigger the pipeline through a partial match, then you should set the trigger condition to File Path contains test.json. This would match the scan results.
• If you only want a fully qualified path, they you can enter File Path has path /user1/fileabc/test.json. This would also match the scan results.

DI-6822/DI-7013: The indicated number of files satisfying the File Browse filter differs from the number of files that display in the current folder (and subfolders) that match the filter. Additionally, the left facet Search displays old values and statistics. There is currently no context awareness between the filters or dependencies.
(No available workaround at this time)

DI-7116: IDS type schemas field names overlap each other when you open several scheme sub-branches.
Workaround: Close some of the neighboring sub-branches to clearly read their IDS type schema field names.

DI-7189: Not all of the related files display on the Workflow Details page or File Info page due to a file display limit currently set to 50.
Workaround: To review all of the related output files, navigate to the Pipeline Processing page.

PIPE-596: The state of pipelines (Active/Disabled) on the Health Monitoring page does not match the state on the Pipeline Design page.
(No available workaround at this time)

PIPE-619: The process list of files with Pipeline [v1/pipeline/process/{pipelineId}] did not fail on a deleted file when it should have.
Workaround: Before you submit a list of files to process with a pipeline, review to makes sure that it does not contain any deleted files.

PIPE-642: The output files list on the File Details Page displays outputs from another pipeline(s).
(No available workaround at this time)

PIPE-736: [Noted in v3.2.0 and v3.1.6] The workflow processing fails if the pipeline name is greater than 256 characters.
Workaround: Ensure that the pipeline name contains less than 256 characters.