View Artifact Information
Artifacts are versioned instances of Intermediate Data Schemas (IDS), protocols, task scripts, or Connectors. To search for specific artifacts and get detailed information about them, you can use the Artifacts option in the TDP's left navigation menu to do any of the following:
- View IDSs and Their Details
- View Protocols and Their Details
- View Task Scripts and Their Details
- View Connectors and Their Details
For more information about each type of pipeline element, see Tetra Data Pipeline Overview. For additional best practices, see Artifacts in the TetraConnect Hub. For access, see Access the TetraConnect Hub.
NOTE
When reviewing artifacts in the default view, the latest version of each artifact is listed. Prior artifact versions are not automatically listed in the default view. To see all of the versions of a specific artifact, you must run a search for the artifact by using the search field. Searches return all of the versions associated with specific artifacts.
View IDSs and Their Details
A TetraScience IDS is applied to raw instrument data or report files to map vendor-specific information (such as a field name) to vendor-agnostic information. The IDS standardizes naming, data type, data range, and data hierarchy.
NOTE
You can view high-level information about different IDSs in Gallery View or List View, and then select individual IDSs to see more details. Each schema entry includes the following information:
- Thumbnail image (shows either a graphic or the first and last letter of the schema's slug)
- IDS name
- IDS namespace
- IDS version
To view available IDSs and to get more information about a specific IDS, do the following:
- In the left navigation menu, choose Artifacts.
- Choose IDS. The list of schemas appears in List View by default. For a gallery view, choose the upper right Gallery View icon.
- To search for specific schemas, do one of the following:
- Enter your search keywords into the upper right search field. The search returns any schema that includes the string you entered in either the IDS slug, name, description, or its associated labels.
-or- - (For List View only) In one or more of the filter fields at the top of each column, enter the values that you want to filter by. You can filter by slug, type (private, common, or client), namespace, version, and last updated date.
- Enter your search keywords into the upper right search field. The search returns any schema that includes the string you entered in either the IDS slug, name, description, or its associated labels.
- To see more information about a specific schema, select the schema. The IDS Details page opens for that schema. This page provides additional information about each schema, such as the available versions, README files, and the last update date and time.
- (Optional) Choose one of the tabs at the top of the page to get more information about the following:
- The Details tab shows the IDS name, description, namespace, slug, related protocols and task scripts, and the last date and time the schema was updated.
- The ReadMe tab shows the README file for the selected IDS version as well as the README files for any of the schema’s associated protocols and task scripts.
- The Labels tab shows the schema’s associated labels that are outlined in the artifact’s
manifest.json
file, if there are any. - The ERD tab shows an interactive entity relationship diagram (ERD) that represents the relational schema for the selected IDS. You can use this diagram to better understand the relationships between an IDS's associated Amazon Athena tables, which can help you create more effective SQL queries. For more information, see TDP Athena SQL Table Structure and Use TDP to Query Athena SQL Tables.
- (Optional) In the ACTIONS section, choose one of the action buttons to run its associated task. For a list of available actions, see the following table. Action buttons are grayed out and unable to be selected if the linked file for that action doesn’t exist.
Actions Available on the IDS Details Page
Action (Button) | Description |
---|---|
Browse IDS Nodes | Provides a graphical representation of the schema |
View elasticsearch.json | Returns the JSON file used to create the Amazon Elasticsearch index |
View manifest.json | Returns the JSON file that contains manifest metadata |
View athena.json | Returns the JSON file used to create the Amazon Athena index |
View schema.json | Returns the schema’s JSON file used in the validation pipeline |
View expected.json | Provides an example of what the resulting JSON should look like once processing is complete |
View build-record.json | Returns the record of the build in JSON format, including the commit, GitHub data, repository, software hazard analysis (SHA), version tag, and the date the schema was created |
View Protocols and Their Details
IMPORTANT
Starting with Tetra Data Platform (TDP) version 3.6.0 and the TetraScience Software Development Kit (SDK) 2.0 release, the
protocol.yml
file format replaces both the previousprotocol.json
andscript.js
file formats when creating self-service Tetra Data pipelines (SSPs).
Protocols define the business logic of your pipeline by specifying the steps and the functions within task scripts that execute those steps.
NOTE
You can view high-level information about different protocols in List View, and then select individual protocols to see more details about each one. Protocol entries include the following information:
- Slug
- Type (private, common, or client)
- Protocol namespace
- Protocol version
- Last updated date
To view available protocols and get more information about a specific protocol, do the following:
- In the left navigation menu, choose Artifacts.
- Choose Protocols. The list of protocols appears in a list view.
- To search for specific protocols, do one of the following:
- Enter your search keywords into the upper right search field. The search returns any protocol that includes the string you entered in either the protocol slug, name, description, or its associated labels.
-or- - In one or more of the filter fields at the top of each column, enter the values that you want to filter by. You can filter by slug, type (private, common, or client), namespace, version, and last updated date.
- Enter your search keywords into the upper right search field. The search returns any protocol that includes the string you entered in either the protocol slug, name, description, or its associated labels.
- To see more information about a specific protocol, select the protocol’s row. The Protocol Details page opens. This page provides additional information about each protocol, such as the available versions, README files, related task scripts, and the last update date and time.
- (Optional) Choose one of the tabs at the top of the page to get more information about the following:
- The Details tab shows the protocol name, description, namespace, slug, related task scripts, and the last date and time the protocol was updated.
- The ReadMe tab shows the README file for the selected protocol version.
- The Labels tab shows the protocol’s associated labels that are outlined in the artifact’s manifest.json file, if there are any.
- (Optional) In the ACTIONS section, choose one of the action buttons to run its associated task. For a list of available actions, see the following table. Action buttons are grayed out and unable to be selected if the linked file for that action doesn’t exist.
Actions Available on the Protocol Details Page
Action (Button) | Description |
---|---|
View manifest.json | Returns the JSON file that contains manifest metadata |
View script.js | Returns the Javascript file for the protocol’s script. This file runs the protocol, which is in the protocol.json file. It declares the inputFile and result variables, and then returns the result. The namespace, slug, and version are part of the result script. |
View protocol.json | Returns the protocol’s JSON file. This file provides information about steps and configurations. Protocol.json references the names of the task scripts associated with each of the steps as well as the configurations that are needed to run the protocol. It displays the protocol’s name, description, step number, step number.slug, step number.description, and step number.type. It also shows the step number.script.slug and step number.script.version variables. |
View protocol.yml | Returns the protocol's YAML file. This file defines the steps and configurations that are used to process data. For more information, see Protocol YAML Files. |
View build-record.json | Returns the record of the build in JSON format, including the commit, GitHub data, repository, software hazard analysis (SHA), version tag, and the date the protocol was created |
View Task Scripts and Their Details
Task scripts contain the code for the business logic needed to process the data in a pipeline. When processing is finished, output files are indexed according to a predefined IDS and stored in the data lake or another specific data target.
NOTE
You can view high-level information about different task scripts in List View, and then select individual scripts to see more details about each one. Task script entries include the following information:
- Slug
- Type (private, common, or client)
- Task script namespace
- Task script version
- Last updated date
To view available task scripts and to get more information about a specific script, do the following:
- In the left navigation menu, choose Artifacts.
- Choose Task Scripts. The list of task scripts appears in a list view.
- To search for specific task scripts, do one of the following:
- Enter your search keywords into the upper right search field. The search returns any script that includes the string you entered in either the protocol slug, name, description, or its associated labels.
-or- - In one or more of the filter fields at the top of each column, enter the values that you want to filter by. You can filter by slug, type (private, common, or client), namespace, version, and last updated date.
- Enter your search keywords into the upper right search field. The search returns any script that includes the string you entered in either the protocol slug, name, description, or its associated labels.
- To see more information about a specific task script, select the script's row. The Task Script Details page opens. This page provides additional information about each task script, such as the available versions, README files, related protocols, and the last update date and time.
- (Optional) Choose one of the tabs at the top of the page to get more information about the following:
- The Details tab shows the task script’s name, description, namespace, slug, related protocols and IDSs, and the last date and time the script was updated.
- The ReadMe tab shows the README file for the selected task script version.
- The Build Logs tab shows the build logs for the selected task script version.
- The Labels tab shows the task script ’s associated labels that are outlined in the artifact’s manifest.json file, if there are any.
- (Optional) In the ACTIONS section, choose one of the action buttons to run its associated task. For a list of available actions, see the following table. Action buttons are grayed out and unable to be selected if the linked file for that action doesn’t exist.
Actions Available on the Task Script Details Page
Action (Button) | Description |
---|---|
View manifest.json | Returns the JSON file that contains manifest metadata |
View build-record.json | Returns the record of the build in JSON format, including the language, slugs, and associated functions as well as other build data |
View Connectors and Their Details
Connector artifacts contain the definition, assets, and code for a Tetra Connector. Connector artifacts have two main files:
manifest.json
: Contains the metadata, specs, and configuration options for the Connector artifactbuild.json
: Describes the Docker image that contains the Connector software
To view available Connector artifacts and to get more information about a specific Connector artifact, do the following:
- In the left navigation menu, choose choose Artifacts.
- Choose Connectors. The list of artifacts appears in List View by default. For a gallery view, choose the upper right Gallery View icon.
- To search for specific Connector artifacts, do one of the following:
- Enter your search keywords into the upper right search field. The search returns any Connector artifacts that includes the string you entered in either the artifact slug, name, description, or its associated labels.
-or- - (For List View only) In one or more of the filter fields at the top of each column, enter the values that you want to filter by. You can filter by slug, type (private, common, or client), namespace, version, and last updated date.
- Enter your search keywords into the upper right search field. The search returns any Connector artifacts that includes the string you entered in either the artifact slug, name, description, or its associated labels.
- To see more information about a specific Connector artifact, select the artifact. The Connector Details page opens for that artifact. This page provides additional information about each Connector artifact, such as the available versions, README files, and the last update date and time.
- (Optional) Choose one of the tabs at the top of the page to get more information about the following:
- The Details tab shows the Connector artifact name, description, namespace, slug, and the last date and time the schema was updated.
- The ReadMe tab shows the README file for the selected Connector artifact version as well as the associated README files, if there are any
- The Labels tab shows the Connector artifact’s associated labels that are outlined in the artifact’s
manifest.json
file, if there are any.
- (Optional) In the ACTIONS section, choose one of the action buttons to run its associated task. For a list of available actions, see the following table. Action buttons are grayed out and unable to be selected if the linked file for that action doesn’t exist.
Actions Available on the Connector Details Page
Action (Button) | Description |
---|---|
View manifest.json | Returns the JSON file that contains manifest metadata |
View build.json | Returns the record of the build in JSON format, including the language, slugs, and associated functions as well as other build data |
Updated 7 months ago