Data consistency is crucial for data stored and accessed within the Tetra Data Platform (TDP). The Files health dashboard on the Health Monitoring page helps you monitor and report on the health of your data files in the TDP. It also provides options to either reprocess selected files, or create Reconciliation Jobs to fix any issues.

Data is currently stored in the following locations:

• Tetra Data Lake (Amazon S3)
• File Registration Database
• System Properties (FileInfo) Service
• Search Indices (Elasticsearch)
• SQL (Amazon Athena)

As an administrator, you can use the Files health dashboard to quickly identify any inconsistencies that exist across these services.

📘

NOTE

To view file events generated by the Tetra File-Log Agent outside of the TDP, select the Integration Events tab on the Health Monitoring Dashboard. For more information, see Monitor Integration Events.

View the Files Health Dashboard

1. Sign in to the TDP with an admin account.
2. In the left navigation menu, choose the hamburger menu icon. Then, choose Health Monitoring. The Health Monitoring page appears.
3. Select the Files tab. The Files health dashboard appears.

Files Health Dashboard Contents

The Files health dashboard has two sections:

• ES CLUSTER HEALTH
• File Failures

ES Cluster Health

The ES CLUSTER HEALTH section at the top of the Files health dashboard shows health status of the Elasticsearch (ES) cluster as reported by Amazon CloudWatch.

Three potential statuses are indicated by the following colors:

• Green—indicates that the ES cluster is healthy and no action is required.
• Yellow—indicates that one or more of the replica shards on the ES cluster are not allocated to a node.
• Red—indicates that at least one primary shard is not allocated to any node.

📘

Yellow or Red Status?

If your ES CLUSTER HEALTH status is yellow or red, please contact your customer success manager (CSM) and/or your AWS IT team.

ES Cluster Storage

To review free and used storage space details for the ES cluster, you can hover over the information icon next to the ES CLUSTER HEALTH status.

Three potential available storage space amounts are indicated for the ES cluster by the following colors:

• Green—indicates greater than 50% free storage space.
• Yellow—indicates less than 30% to 50% free storage space.
• Red— indicates less than 30% free storage space.

File Failures

The File Failures section shows file event failures only. You can view general information about each file event failure in the list provided, or select a specific file to see the event's error details.

You can search for specific files or sort the list by doing the following:

• To search for a specific file, enter a file name or file ID in the search field.
• To filter the list by error code types, select an error code type from the Filter Error Codes drop-down list.
• To sort the list by component type, select one of the following options that are listed below Component Type at the top of the list: All, FileInfo, Athena, or Elasticsearch.

File Failures Information

The File Failures list provides the following information:

View a Specific File's Failure Details

To see the details of a specific file failure, select the file name from the File Processing Failures list. Additional rows appear that provide more information about the failure, the error message it returned, and the event's associated logs.

To see the logs for a specific event, select the arrow icon in the RAW EVENT column for that event.

Reprocess Selected Files, Create a Reconciliation Job, or View an Existing Job

At the top right of the Files health monitoring dashboard, you can select any of the following actions:

• View Jobs—opens the Reconciliation Jobs page, where you can either view your organization's existing jobs or create a new one.
• Create a Reconciliation Job—opens a dialog that provides configuration options for creating a Reconciliation Job.
• Reprocess Selected files—allows you to run a reconciliation job on any specific file failures that you've selected.

📘

NOTE

You can also create a Reconciliation Job from the Reconciliation Jobs page. For more information, see Monitor and Reprocess Files by Using the Reconciliation Jobs Page.

Monitor a Reconciliation Job's Status

To monitor a Reconciliation Job's status, go to the Reconciliation Jobs page by doing the following:

1. In the left navigation pane, choose the hamburger menu icon.
2. Choose Bulk Actions. Then, choose Reconciliation Jobs. The Reconciliation Jobs page appears and displays a list of all your active and inactive Reconciliation Jobs, which includes the following information:

• STATE—shows the job’s status
• NAME—shows the job’s name
• COMPLETION—shows how much of the job has been processed (measured as a percentage)
• STARTED—shows the date and time the job started processing
• COMPLETED—shows the date and time the job completed
• ERRORS—shows any errors that occurred during the job
• COMPONENT—shows the storage type (fileinfoToAthena, fileInfoToEs, or s3ToFileInfo)
• INFO—opens a Bulk Processing Job Details dialog that shows additional information about the job, including the JOB ID, TYPE, and FILE STATUS for each file the job processed.

📘

NOTE

To create a new Reconciliation Job from the Reconciliation Jobs page, select the upper right Create Reconciliation Job button. For more information, see Monitor and Reprocess Files by Using the Reconciliation Jobs Page.