Event Subscriptions

Event subscriptions provide a way to programmatically notify downstream systems of key events in your Tetra Data Platform (TDP) environment. By subscribing to specific TDP events, you can then automatically trigger custom apps and business processes to accomplish follow-on actions, such as the following:

  • Notify a custom app when a file is uploaded to the TDP from a specific instrument type
  • Alert scientists when a file is available for search
  • Programmatically monitor Tetra Agent health
  • Fuel workflow operational metrics and custom dashboards

Create an Event Subscription

Event subscriptions are delivered through Amazon EventBridge. To create an event subscription, you must first create an EventBridge event bus in your AWS account. The event bus contains rules that match incoming events and send them to targets for processing. Then, you can select the TDP events that you want the event bus to receive by configuring an event subscription on the Event Subscriptions page.

Step 1: Set Up an EventBridge Event Bus

To create an Amazon EventBridge event bus, do the following:

  1. Sign in to the Amazon EventBridge console.

  2. Create a custom event bus. For instructions, see Creating an Amazon EventBridge event bus in the AWS documentation.

    🚧

    IMPORTANT

    For subscribing to events across AWS accounts, make sure that you allow the AWS account where the TDP is running to put events into your target AWS account. To set up the required permissions, enter the following AWS Identity and Access Management (IAM) policy into the Resource-based policy section:

    {
     "Version": "2012-10-17",
     "Statement": [{
       "Sid": "AllowAccountToPutEvents",
      "Effect": "Allow",
       "Principal": {
         "AWS": "arn:aws:iam::<AWS_ACCOUNT_ID_OF_TETRASCIENCE_DEPLOYMENT>:root"
       },
       "Action": "events:PutEvents",
       "Resource": "arn:aws:events:<MY_AWS_REGION>:<MY_AWS_ACCOUNT_ID>:event-bus/<MY_EVENT_BUS_NAME>"
     }]
    }

    To get the AWS account ID for your TDP deployment (<AWS_ACCOUNT_ID_OF_TETRASCIENCE_DEPLOYMENT>), contact your customer success manager (CSM).

  3. Create rules for the event bus that specify what actions to take when EventBridge receives an event that matches the event patterns in the rule. For instructions, see Create a rule in Amazon EventBridge and Creating event patterns in the AWS documentation. For specific examples, see Event Pattern Examples.

  4. On the Event bus details pane, copy the Event bus ARN (Amazon Resource Name). You will need the full event bus ARN to set up the subscription to the event bus in the TDP user interface.

Step 2: Configure an Event Subscription in the TDP

To select the TDP events that you want your event bus to receive, do the following:

  1. Sign in to the TDP as a user with Admin permissions.
  2. In the left navigation menu, choose Administration. Then, choose Event Subscriptions. The Event Subscriptions page appears.
  3. Choose Create Event Subscription. The Create Event Subscription dialog appears.
  4. For SUBSCRIPTION NAME, enter a name for the subscription.
  5. To enable the subscription immediately after it's created, move the ENABLE SUBSCRIPTION toggle to the right. To keep the subscription inactive, move the toggle to the left.
  6. For EVENTS, select the events that you want to subscribe to.
  7. For EVENT BUS NAME enter the full Amazon Resource Name (ARN) of the EventBridge event bus that you created.
  8. Choose Save.

Event Payloads

All TDP events delivered through EventBridge share a common JSON envelope structure. The detail object within the envelope contains the event-specific payload, which varies by event type.

Common EventBridge Envelope

Every event delivered to your event bus uses the following outer structure, regardless of event type:

📘

NOTE

Each event's payload includes a "detail-type" key/value pair based on the AWS resource that generates the event (for example, "Event from aws:kinesis" for events routed through Amazon Kinesis Data Streams).

{
    "version": "0",
    "id": "<unique-aws-id>",
    "detail-type": "Event from aws:kinesis",
    "source": "<event-subscription-pipe-name>",
    "account": "<aws-destination-account-id>",
    "time": "2023-12-12T15:25:26Z",
    "region": "us-east-2",
    "resources": [],
    "detail": {
        "type": "<tetrascience-event-type>",
        "orgSlug": "<your-org-slug>",
        "record": { ... }
    }
}
FieldDescription
versionEvent version number provided by EventBridge (always "0")
idUnique AWS event ID
detail-typeAWS resource type that generated the event
sourceName of the TDP event subscription pipe
accountAWS destination account ID
timeEvent timestamp in ISO 8601 format (UTC)
regionAWS region
resourcesUnused (always empty)
detail.typeTDP event type (for example, tetrascience.file.WorkflowSucceeded.v1)
detail.orgSlugOrg slug of the TDP organization where the event was generated
detail.recordEvent-specific payload — varies by event type (see sections below)

File Event Payloads

File events are emitted during the file lifecycle in the TDP. The detail.record object varies depending on whether the event relates to file registration, search/SQL indexing, or pipeline workflows.

Registration Events

RegistrationSucceeded.v1 fires once when a file is first written to TDP, and again when a new file version is uploaded. It does not re-fire when labels, tags, or metadata are updated on an existing file version.

The following events are emitted when a file is registered (or fails to register) in the TDP.

Event TypeDescription
tetrascience.file.RegistrationSucceeded.v1A file is successfully registered in the TDP
tetrascience.file.RegistrationFailed.v1A file registration fails

Example detail.record for RegistrationSucceeded

{
    "file": {
        "fileId": "5bbfefad-9f9f-4677-94f5-fbf3a6bf88bd",
        "fileKey": "your-org/6f49fk393kdk/RAW/instrument_output.raw",
        "version": "X846ooooqvx493kd0fkd",
        "bucket": "ts-platform-datalake",
        "type": "s3file",
        "category": "RAW",
        "traceId": "ba683c67-7518-4c71-9690-0da82f2e92b3",
        "source": {
            "type": "empower"
        },
        "metadata": {},
        "tags": [],
        "labels": [
            {
                "name": "project_id",
                "value": "proj12345",
                "created_at": "2023-12-10T13:25:26Z",
                "created_by": null,
                "mechanism_type": "agent",
                "mechanism_id": "55953bf0-acb6-4a45-beb6-cb30b33d4941"
            }
        ]
    },
    "createdAt": "2023-12-10T13:25:26Z"
}

Search and SQL Indexing Events

These events fire on initial file registration and whenever labels, tags, or metadata are updated on an existing file. If you are building an integration that writes labels back to TDP in response to these events, subscribe to RegistrationSucceeded.v1 instead to avoid an infinite loop.

The following events are emitted when a file is indexed (or fails to index) in OpenSearch or Amazon Athena.

Event TypeDescription
tetrascience.file.SearchIndexingSucceeded.v1A file is indexed in OpenSearch
tetrascience.file.SearchIndexingFailed.v1OpenSearch indexing for a file fails
tetrascience.file.SQLIndexingSucceeded.v1A file is indexed in Amazon Athena
tetrascience.file.SQLIndexingFailed.v1Amazon Athena indexing for a file fails

Example detail.record for SearchIndexingSucceeded

{
    "file": {
        "fileId": "5bbfefad-9f9f-4677-94f5-fbf3a6bf88bd",
        "fileKey": "your-org/6f49fk393kdk/IDS/test_file/0.json",
        "version": "X846ooooqvx493kd0fkd",
        "bucket": "ts-platform-datalake",
        "type": "s3file",
        "category": "IDS",
        "traceId": "ba683c67-7518-4c71-9690-0da82f2e92b3",
        "source": {
            "type": "empower"
        },
        "metadata": {},
        "tags": [],
        "labels": [
            {
                "name": "project_id",
                "value": "proj12345",
                "created_at": "2023-12-10T13:25:26Z",
                "created_by": null,
                "mechanism_type": "agent",
                "mechanism_id": "55953bf0-acb6-4a45-beb6-cb30b33d4941"
            }
        ]
    },
    "createdAt": "2023-12-10T13:25:26Z"
}

Workflow Events

The following events are emitted during the pipeline workflow lifecycle.

Event TypeDescription
tetrascience.file.WorkflowCreated.v1A pipeline workflow is created
tetrascience.file.WorkflowInProgress.v1A pipeline workflow is triggered and running
tetrascience.file.WorkflowSucceeded.v1A pipeline workflow completes successfully
tetrascience.file.WorkflowFailed.v1A pipeline workflow fails
tetrascience.file.WorkflowCanceled.v1A pipeline workflow is canceled

Example detail.record for WorkflowSucceeded

{
    "file": {
        "fileId": "5bbfefad-9f9f-4677-94f5-fbf3a6bf88bd",
        "fileKey": "your-org/6f49fk393kdk/IDS/test_file/0.json",
        "version": "X846ooooqvx493kd0fkd",
        "bucket": "ts-platform-datalake",
        "type": "s3file",
        "category": "IDS",
        "traceId": "ba683c67-7518-4c71-9690-0da82f2e92b3",
        "source": {
            "type": "empower"
        },
        "metadata": {},
        "tags": [],
        "labels": [
            {
                "name": "project_id",
                "value": "proj12345"
            }
        ]
    },
    "createdAt": "2023-12-10T13:25:26Z",
    "workflowId": "4465a573-5e65-40bf-b2f5-b9a1b3d170b2",
    "pipelineId": "df1f9835-376c-4899-bd93-17e116723e85"
}

detail.record Field Reference for File Events

FieldTypePresent InDescription
file.fileIdstringAll file eventsUnique file identifier in the TDP
file.fileKeystringAll file eventsFull S3 key path for the file
file.versionstringAll file eventsS3 object version ID
file.bucketstringAll file eventsS3 bucket where the file is stored
file.typestringAll file eventsFile storage type (for example, s3file)
file.categorystringAll file eventsFile category in the TDP (for example, RAW, IDS)
file.traceIdstringAll file eventsTrace ID linking related file operations
file.source.typestringAll file eventsData source type (for example, empower)
file.metadataobjectAll file eventsFile metadata key-value pairs
file.tagsarrayAll file eventsFile tags
file.labelsarrayAll file eventsFile labels. Each label object includes name, value, created_at, created_by, mechanism_type, and mechanism_id.
createdAtstringAll file eventsTimestamp of when the event was created (ISO 8601 UTC)
workflowIdstringWorkflow events onlyID of the TDP workflow
pipelineIdstringWorkflow events onlyID of the pipeline

Agent Event Payloads

Agent events use the same common EventBridge envelope as file events. The detail.record object for agent events contains agent-specific fields.

For the full list of events and their detail.record payloads, see:

  • Common Agent Events — Events shared by all Tetra Agents (agentStarted, agentStopped, heartbeat)
  • Tetra File-Log Agent Events — Events specific to the File-Log Agent (fileScanned, fileUploadCompleted, fileArchiveCompleted, and others)
📘

NOTE

The agent event pages linked above show only the event-specific data within detail.record. The full EventBridge payload wraps this data in the common envelope structure shown above.

Event Types Reference

Event subscriptions currently support the following event types, grouped by category. For payload details, see File Event Payloads and Agent Event Payloads.

Event TypeCategoryDescription
tetrascience.file.RegistrationSucceeded.v1RegistrationA file is successfully registered in the TDP
tetrascience.file.RegistrationFailed.v1RegistrationA file registration fails
tetrascience.file.SearchIndexingSucceeded.v1IndexingA file is indexed in OpenSearch
tetrascience.file.SearchIndexingFailed.v1IndexingOpenSearch indexing for a file fails
tetrascience.file.SQLIndexingSucceeded.v1IndexingA file is indexed in Amazon Athena
tetrascience.file.SQLIndexingFailed.v1IndexingAmazon Athena indexing for a file fails
tetrascience.file.WorkflowCreated.v1WorkflowA pipeline workflow is created
tetrascience.file.WorkflowInProgress.v1WorkflowA pipeline workflow is triggered and running
tetrascience.file.WorkflowSucceeded.v1WorkflowA pipeline workflow completes successfully
tetrascience.file.WorkflowFailed.v1WorkflowA pipeline workflow fails
tetrascience.file.WorkflowCanceled.v1WorkflowA pipeline workflow is canceled
tetrascience.agents.common.agentStarted.v1AgentAn Agent is started
tetrascience.agents.common.agentStopped.v1AgentAn Agent is stopped
tetrascience.agents.common.heartbeat.v1AgentOccurs on a regular interval to indicate the agent is running
tetrascience.agents.filelog.archiveFileDeleteCompleted.v1File-Log AgentA file is successfully deleted from the archive location
tetrascience.agents.filelog.archiveFileDeleteFailed.v1File-Log AgentAn error occurs during archive file deletion
tetrascience.agents.filelog.fileArchiveCompleted.v1File-Log AgentA file is successfully archived
tetrascience.agents.filelog.fileArchiveFailed.v1File-Log AgentAn error occurs during file archival
tetrascience.agents.filelog.fileArchiveSkipped.v1File-Log AgentFile archiving for a file is skipped because of a missing checksum for the source file
tetrascience.agents.filelog.fileArchiveStarted.v1File-Log AgentFile archiving begins for a file
tetrascience.agents.filelog.fileScanned.v1File-Log AgentA file has been scanned and identified for upload
tetrascience.agents.filelog.fileUploadCompleted.v1File-Log AgentA file upload has completed successfully
tetrascience.agents.filelog.fileUploadFailed.v1File-Log AgentAn error occurs before or during the upload of a file
tetrascience.agents.filelog.fileUploadStarted.v1File-Log AgentA file upload begins
tetrascience.agents.filelog.pathValidationFailed.v1File-Log AgentA file's path validation fails (is inaccessible) before scan operation starts
tetrascience.agents.filelog.scanCompleted.v1File-Log AgentA scan operation has completed
tetrascience.agents.filelog.scanError.v1File-Log AgentAn error occurs during a scan
tetrascience.agents.filelog.scanStarted.v1File-Log AgentA scan of a given path starts

Event Pattern Examples

The following are example EventBridge event patterns for common event subscription use cases.

For more information about how to create custom event patterns, see Amazon EventBridge event patterns in the AWS documentation.

📘

NOTE

You can test event patterns by using the EventBridge Sandbox. For more information, see Testing an event pattern using the EventBridge Sandbox in the AWS documentation.

Alert When any File in Your Organization is Available for Search

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.SearchIndexingSucceeded.v1"]
  }
}

Alert When a Workflow for a specific Pipeline Completed Successfully

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.WorkflowSucceeded.v1"],
    "record": {
      "pipelineId": ["9137597a-4a69-45ec-815a-3fc40fc1e6d2"]
    }
  }
}

Alert When an IDS File with the Label {software: Empower} in your organization is Available for Search

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.SearchIndexingSucceeded.v1"],
    "record": {
      "file": {
        "category": ["IDS"],
        "labels": {
          "name": ["software"],
          "value": ["Empower"]
        }
      }
    }
  }
}

Monitor Agent Heartbeat Events in Your Organization

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.agents.common.heartbeat.v1"]
}

Monitor for File-Log Agent Failure Events in Your Organization

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.agents.filelog.archiveFileDeleteFailed.v1", "tetrascience.agents.filelog.fileArchiveFailed.v1", "tetrascience.agents.filelog.fileUploadFailed.v1", "tetrascience.agents.filelog.pathValidationFailed.v1", "tetrascience.agents.filelog.scanError.v1"]
}

Fuel Workflow Operational Metrics and Custom Dashboards Across Multiple Organizations

{
  "detail": {
    "orgSlug": ["your-org-1", "your-org-2"],
    "type": [{
      "prefix": "tetrascience.file.Workflow"
    }]
 }
📘

NOTE

For this setup to work, you must have access to both "your-org-1" and "your-org-2". You must also create a separate event subscription for each organization, and both event subscriptions must point to the event bus that this event rule belongs to.

Filter File Events by File Extension

The following event pattern uses EventBridge suffix matching with equals-ignore-case to match only SearchIndexingSucceeded events where the file key ends with .tiff or .nd2, regardless of case. The record.file.fileKey field is available for all file.* event types.

{
  "detail": {
    "orgSlug": ["your-org"],
    "type": ["tetrascience.file.SearchIndexingSucceeded.v1"],
    "record": {
      "file": {
        "fileKey": [{
          "suffix": {
            "equals-ignore-case": ".tiff"
          }
        }, {
          "suffix": {
            "equals-ignore-case": ".nd2"
          }
        }]
      }
    }
  }
}
📘

NOTE

This event pattern filters events after they arrive on your event bus. It controls which events are routed to downstream targets (such as Lambda, SQS, or SNS), but does not reduce the number of events sent to your bus. TDP event subscriptions do not currently support server-side content filtering — all events of a selected type are sent to your bus regardless of content. Because EventBridge charges per event ingested, not per event matched, consider consolidating event buses where possible to avoid duplicate ingestion charges.

Documentation Feedback

Do you have questions about our documentation or suggestions for how we can improve it? Start a discussion in TetraConnect Hub. For access, see Access the TetraConnect Hub.

📘

NOTE

Feedback isn't part of the official TetraScience product documentation. TetraScience doesn't warrant or make any guarantees about the feedback provided, including its accuracy, relevance, or reliability. All feedback is subject to the terms set forth in the TetraConnect Hub Community Guidelines.