Event Subscriptions

Event subscriptions provide a way to programmatically notify downstream systems of key events in your Tetra Data Platform (TDP) environment. By subscribing to specific TDP events, you can then automatically trigger custom apps and business processes to accomplish follow-on actions, such as the following:

  • Notify a custom app when a file is uploaded to the TDP from a specific instrument type
  • Alert scientists when a file is available for search
  • Programmatically monitor Tetra Agent health
  • Fuel workflow operational metrics and custom dashboards

Create an Event Subscription

Event subscriptions are delivered through Amazon EventBridge. To create an event subscription, you must first create an EventBridge event bus in your AWS account. The event bus contains rules that match incoming events and send them to targets for processing. Then, you can select the TDP events that you want the event bus to receive by configuring an event subscription on the Event Subscriptions page in the TDP user interface.

Step 1: Set Up an EventBridge Event Bus

To create an Amazon EventBridge event bus, do the following:

  1. Sign in to the Amazon EventBridge console.

  2. Create a custom event bus. For instructions, see Creating an Amazon EventBridge event bus in the AWS documentation.

    🚧

    IMPORTANT

    For subscribing to events across AWS accounts, make sure that you allow the AWS account where the TDP is running to put events into your target AWS account. To set up the required permissions, enter the following AWS Identity and Access Management (IAM) policy into the Resource-based policy section:

    {
     "Version": "2012-10-17",
     "Statement": [{
       "Sid": "AllowAccountToPutEvents",
      "Effect": "Allow",
       "Principal": {
         "AWS": "arn:aws:iam::<AWS_ACCOUNT_ID_OF_TETRASCIENCE_DEPLOYMENT>:root"
       },
       "Action": "events:PutEvents",
       "Resource": "arn:aws:events:<MY_AWS_REGION>:<MY_AWS_ACCOUNT_ID>:event-bus/<MY_EVENT_BUS_NAME>"
     }]
    }
    
  3. Create rules for the event bus that specify what actions to take when EventBridge receives an event that matches the event patterns in the rule. For instructions, see Create a rule in Amazon EventBridge and Creating event patterns in the AWS documentation. For specific examples, see Event Pattern Examples.

  4. On the Event bus details pane, copy the Event bus ARN (Amazon Resource Name). You will need the full event bus ARN to set up the subscription to the event bus in the TDP user interface.

Step 2: Configure an Event Subscription in the TDP

To select the TDP events that you want your event bus to receive, do the following:

  1. Sign in to the TDP as a user with Admin permissions.
  2. In the left navigation menu, choose Administration. Then, choose Event Subscriptions. The Event Subscriptions page appears.
  3. Choose Create Event Subscription. The Create Event Subscription dialog appears.
  4. For SUBSCRIPTION NAME, enter a name for the subscription.
  5. To enable the subscription immediately after it's created, move the ENABLE SUBSCRIPTION toggle to the right. To keep the subscription inactive, move the toggle to the left.
  6. For EVENTS, select the events that you want to subscribe to.
  7. For EVENT BUS NAME enter the full Amazon Resource Name (ARN) of the EventBridge event bus that you created.
  8. Choose Save.

Event Types

Event subscriptions currently support the following event types.

Event TypeDescription
tetrascience.file.RegistrationSucceeded.v1A file is successfully registered in the TDP
tetrascience.file.RegistrationFailed.v1A file TDP registration fails
tetrascience.file.SearchIndexingSucceeded.v1A file is indexed in OpenSearch
tetrascience.file.SearchIndexingFailed.v1OpenSearch indexing for a file fails
tetrascience.file.SQLIndexingSucceeded.v1A file is indexed in Amazon Athena
tetrascience.file.SQLIndexingFailed.v1Amazon Athena indexing for a file fails
tetrascience.file.WorkflowCreated.v1Pipeline workflow is created
tetrascience.file.WorkflowInProgress.v1Pipeline workflow is triggered and running
tetrascience.file.WorkflowSucceeded.v1A triggered pipeline workflow completes successfully
tetrascience.file.WorkflowFailed.v1A triggered pipeline workflow fails
tetrascience.file.WorkflowCanceled.v1A triggered pipeline workflow is canceled
tetrascience.agents.common.agentStarted.v1An Agent is started
tetrascience.agents.common.agentStopped.v1An Agent is stopped
tetrascience.agents.common.heartbeat.v1Occurs on a regular interval to indicate the agent is running
tetrascience.agents.filelog.archiveFileDeleteCompleted.v1(File-Log Agent event) A file is successfully deleted from the archive location
tetrascience.agents.filelog.archiveFileDeleteFailed.v1(File-Log Agent event) An error occurs during archive file deletion
tetrascience.agents.filelog.fileArchiveCompleted.v1(File-Log Agent event) A file is successfully archived
tetrascience.agents.filelog.fileArchiveFailed.v1(File-Log Agent event) An error occurs during file archival
tetrascience.agents.filelog.fileArchiveSkipped.v1(File-Log Agent event) File archiving for a file is skipped because of a missing checksum for the source file. This occurs when the file was originally uploaded by a previous version of the File-Log Agent, which didn't calculate and store the checksum at upload time.
tetrascience.agents.filelog.fileArchiveStarted.v1(File-Log Agent event) File archiving begins for a file
tetrascience.agents.filelog.fileScanned.v1(File-Log Agent event) A file has been scanned and identified for upload. This event is raised after the file has been persisted to the local SQLite database.
tetrascience.agents.filelog.fileUploadCompleted.v1(File-Log Agent event) A file upload has completed successfully
tetrascience.agents.filelog.fileUploadFailed.v1(File-Log Agent event) An error occurs before or during the upload of a file
tetrascience.agents.filelog.fileUploadStarted.v1(File-Log Agent event) A file upload begins
tetrascience.agents.filelog.pathValidationFailed.v1(File-Log Agent event) A file's path validation fails (is inaccessible) before scan operation starts
tetrascience.agents.filelog.scanCompleted.v1(File-Log Agent event) A scan operation (identified by scanId) has completed. fileCount indicates the number of files found.
tetrascience.agents.filelog.scanError.v1(File-Log Agent event) An error occurs during a scan. For example, an IO exception occurs when trying to access a directory in a path.
tetrascience.agents.filelog.scanStarted.v1(File-Log Agent event) A scan of a given path starts. This particular scan operation has a unique identifier associated with it, which is included in the event as scanId.

Event Pattern Examples

The following are example EventBridge event patterns for common event subscription use cases.

For more information about how to create custom event patterns, see Amazon EventBridge event patterns in the AWS documentation.

πŸ“˜

NOTE

You can test event patterns by using the EventBridge Sandbox. For more information, see Testing an event pattern using the EventBridge Sandbox in the AWS documentation.

Alert When any File in Your Organization is Available for Search

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.SearchIndexingSucceeded.v1"]
  }
}

Alert When an IDS File with the Label {software: Empower} in your organization is Available for Search

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.SearchIndexingSucceeded.v1"],
    "record": {
      "file": {
        "category": ["IDS"],
        "labels": {
          "name": ["software"],
          "value": ["Empower"]
        }
      }
    }
  }
}

Monitor Agent Heartbeat Events in Your Organization

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.agents.common.heartbeat.v1"]
}

Monitor for File-Log Agent Failure Events in Your Organization

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.agents.filelog.archiveFileDeleteFailed.v1", "tetrascience.agents.filelog.fileArchiveFailed.v1", "tetrascience.agents.filelog.fileUploadFailed.v1", "tetrascience.agents.filelog.pathValidationFailed.v1", "tetrascience.agents.filelog.scanError.v1"]
}

Fuel Workflow Operational Metrics and Custom Dashboards Across Multiple Organizations

{
  "detail": {
    "orgSlug": ["your-org-1", "your-org-2"],
    "type": [{
      "prefix": "tetrascience.file.Workflow"
    }]
 }

πŸ“˜

NOTE

For this setup to work, you must have access to both "your-org-1" and "your-org-2". You must also create a separate event subscription for each organization, and both event subscriptions must point to the event bus that this event rule belongs to.