How to install or upgrade the connector

Installing the TetraScience IDBS E-Workbook Connector involves four main steps: (1) creating a Docker volume, (2) downloading the Docker image, (3) creating the container with the appropriate volume mapping, and (4) starting the container. To do this, the system must have Docker and the AWS CLI v2 installed.

📘

Host Connector on Data Hub

We recommend that the connector be installed on a TetraScience Data Hub. Both Docker and the AWS CLI v2 will be already installed if the system is also a TetraScience DataHub.

📘

Use Privileged Shell for CLI Commands

All of the commands should be done in a privileged shell.

macOS: If you are using an Administrator account, no additional action is required.

Linux: Run sudo su - to get a root login shell in which to execute the commands.

Step 1: Create volume

Although optional, we recommend using a Docker volume to store the SQLite databases so that configuration is automatically preserved during connector upgrades.

📘

If Upgrading instead of Fresh Install...

If you are upgrading the connector, or for some reason your intended Docker volume already exists, skip to the Step 2: Download image section below.

To create a volume, run this shell command:

docker volume create --driver local idbs_connector_config

• This will create a Docker volume called idbs_connector_config, which is typically located at /var/lib/docker/volumes/idbs_connector_config in a standard Docker installation on Linux.
• When we create the container, we will map this volume to /app/config within the container.
• The connector uses this folder to store configuration, state, and logs in SQLite database files.

Step 2: Download image

Released versions of the IDBS connector are available to customers via Egnyte starting with version v1.0.0. Contact your TetraScience Representative for access. The following screenshot of the IDBS E-WorkBook Connector folder in Egnyte is presented below for reference:

1. Navigate to the IDBS Connector Installer page on Egnyte and select the target version of the IDBS Connector.
2. Download the tarfile containing the Docker image to the host server.
3. Extract the Docker image from the tarfile in the local command line terminal:

docker load --input ts-idbs-connector-v1.1.0.tar

4. The download and image creation was successful if Docker can present the IDBS Connector in its list of images. Run the following command in local terminal to verify:

docker image ls -a

Step 3a: Preparing for Migrations

📘

Skip to Step 3b for Fresh Install

1. Create an export of your Connector’s existing configuration. This export will remove sensitive fields such as SSO Client Secret, SSL Certificate and Key, TDP Service User Token and Agent ID, etc. All DataWeave scripts will be preserved for future import.

2. We recommend that you also make a copy of your existing Docker Volume, or at minimum make a copy of the database file config.db
3. In the next section you will be creating a new container with the new Connector version. If you have an existing connector running, you need to stop that container. You can find the name and container ID of the running containers via:

docker container ls -a

4. Stop the container of interest:

docker container stop <container name or container ID>

docker stop idbs-connector

5. Rename the container that you have just stopped - this will prevent any issues due to duplicate naming from launching the new Connector.

docker rename <container name or container ID> <new name>

docker rename idbs-connector idbs-connector-rollback

Step 3b: Create Container

To create the Docker container, you will use the following arguments.

• -p 3003:8080: forward traffic incoming on port 3003 on the host to port 8080 in the container
• --name=idbs_connector: give the container a specific name of idbs_connector
• -v idbs_connector_config:/app/config: map /app/config in the container to use the Docker volume idbs_connector_config set up in a previous step.
• idbs-connector:v1.1.0 This is the image to base the container on. Your version number may differ.

Create the Docker container with this shell command:

docker create -p 3003:8080 --name=idbs_connector \
--memory=2g -v idbs_connector_config:/app/config idbs-connector:v1.1.0

🚧

Container Memory Limit

The command above includes a memory limit of 2GiB for the container. The intent of the memory limit is to avoid the connector causing out-of-memory behavior for the whole datahub and possibly impacting other connectors. Depending on the memory available on your datahub and what other connectors you are using, you can increase this limit to 4GiB or potentially remove it altogether.

Step 4: Start container

1. To start the container you created:

docker start idbs_connector

• The connector may take up to 15 seconds to become available once the container is started.

2. If this is the first time starting the connector, you should be able to open up a browser to http://<hostname>:3003/ and set up the initial configuration.

• If you have previously configured the connector to use SSL, then go to https://<hostname>:3003/.
• Note that <hostname> is localhost for local development or local access from the server.

Step 5: Migrations clean up

📘

Skip this step if you did not upgrade or perform migrations

If upgrade or migrations went well and as expected, and you have confirmed all configuration and scripts persisted, delete the old connector’s container as a clean up:

docker rm <container name or container ID>

docker rm idbs-connector-rollback