API DocumentationProduct Documentation
Product Documentation

Troubleshooting Tetra Data Hub and Docker Connectivity Issues

Suggest Edits

This topic describes how to troubleshoot docker issues that may occur when you install the Tetra Data Hub.

Tetra Data Hub and Docker Host

After you connect as root, you can check the Docker Engine status, list the docker images in the Tetra Data Hub, remove docker images, and list the docker containers running the Hub.

Log In as Root

Before you can view information about the Docker Engine, images, and containers, you must log in as root.

To log in as root:

  1. Open a Shell tool on the machine that the Tetra Data Hub is installed on.
  2. At the command prompt, enter sudo su -: 
[ec2-user@ip-10-x-x-x tmp]$ sudo su -
Last login: Tue Sep  7 15:07:12 UTC 2021 from 10-x-x-x on pts/0
[root@ip-10-x-x-x ~]#

Docker Engine Status

To check the Docker Engine status, enter systemctl status docker at the command prompt:

[root@ip-10-x-x-x ~]# systemctl status docker
docker.service - Docker Application Container Engine
   Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
  Drop-In: /etc/systemd/system/docker.service.d
           └─aws.conf
   Active: active (running) since Tue 2021-09-07 15:39:29 UTC; 2h 13min ago
     Docs: https://docs.docker.com
 Main PID: 16811 (dockerd)
    Tasks: 34
   Memory: 285.9M
   CGroup: /system.slice/docker.service
           ├─16811 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
           ├─17393 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8443 -container-ip 172.x.x.x -container-port 8443
           ├─17399 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 8443 -container-ip 172.x.x.x -container-port 8443
           ├─17411 /usr/bin/docker-proxy -proto tcp -host-ip 0.0.0.0 -host-port 8080 -container-ip 172.x.x.x -container-port 8080
           └─17417 /usr/bin/docker-proxy -proto tcp -host-ip :: -host-port 8080 -container-ip 172.x.x.x -container-port 8080

Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.432577050Z" level=warning msg="Your kernel does not support cg>
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.432819910Z" level=info msg="Loading containers: start."
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.580606753Z" level=info msg="Default bridge (docker0) is assign>
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.637435731Z" level=info msg="Loading containers: done."
Sep 07 15:39:29 ip-10-x-x-5.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.658053672Z" level=info msg="Docker daemon" commit=75249d8 grap>
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.658220494Z" level=info msg="Daemon has completed initializatio>
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal systemd[1]: Started Docker Application Container Engine.
Sep 07 15:39:29 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:29.675263025Z" level=info msg="API listen on /var/run/docker.sock"
Sep 07 15:39:32 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:32.005128000Z" level=warning msg="reference for unknown type: " d>
Sep 07 15:39:49 ip-10-x-x-x.us-east-2.compute.internal dockerd[16811]: time="2021-09-07T15:39:49.595011866Z" level=info msg="Configured log driver does not sup>

List Docker Images

To list the docker images in the Data Hub (the host), enter docker images -a at the command prompt:

[root@ip-10-x-x-x ~]# docker images -a
REPOSITORY                                                               TAG       IMAGE ID       CREATED      SIZE
xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/data-connector-generic-v2   <none>    xxxxxxxxxxxx   6 days ago   245MB

Remove Docker Images

To verify connectivity and authentication to the AWS ECR (docker registry), it may be helpful to remove docker images and force a pull of a fresh docker image from the registry. To remove docker images (locally), enter docker rmi -f <image_name/id> at the command prompt. 

docker rmi -f <image_name/id>

List Docker Containers

To list docker containers running in the docker host (Data Hub), enter docker ps -a at the command prompt: 

[root@ip-10-x-x-x ~]# docker ps -a
CONTAINER ID   IMAGE                                                                    COMMAND                  CREATED         STATUS         PORTS                                                                                  NAMES

xxxxxxxxxxxx   xxxxxxxxxxxxx.dkr.ecr.us-east-1.amazonaws.com/data-connector-generic-v2   "docker-entrypoint.s…"   8 seconds ago   Up 7 seconds   0.0.0.0:8080->8080/tcp, :::8080->8080/tcp, 0.0.0.0:8443->8443/tcp, :::8443->8443/tcp   nervous_wright

Connector/Docker Container

You can use these commands to enter the running docker container (connector), test the connectivity to S3, install network tools for debugging, and connect to Cloudwatch and S3. You can also verify that your DNS can resolve to the S3 endpoints, and how to view the container's environment variables.

Enter the Running Docker Container

To enter the running docker container (connector), enter docker exec -it /bin/bash at the command prompt on the docker host (Tetra Data Hub): 

[root@ip-10-x-x-x ~]# docker exec -it nervous_wright /bin/bash
bash-5.0#

Test Container Connectivity to S3

To test the container connectivity to S3, enter ping s3.amazonaws.com at the command prompt:

bash-5.0# ping s3.amazonaws.com
PING s3.amazonaws.com (52.217.199.216): 56 data bytes
64 bytes from 52.x.x.x: seq=0 ttl=30 time=12.349 ms
64 bytes from 52.x.x.x: seq=1 ttl=30 time=11.744 ms
64 bytes from 52.x.x.x: seq=2 ttl=30 time=11.721 ms
64 bytes from 52.x.x.x: seq=3 ttl=30 time=11.768 ms
^C
--- s3.amazonaws.com ping statistics ---
4 packets transmitted, 4 packets received, 0% packet loss
round-trip min/avg/max = 11.721/11.895/12.349 ms

📘

Firewall Settings and Connectivity

If your firewall settings block ping, it may not fully validate the connectivity required for the Tetra Data Hub/Connector to operate. However even without validation, the Tetra Data Hub/Connector may still be able to operate normally if it meets the other requirements described in Tetra Data Hub Allow List Endpoints.

Install Network Tools for Debugging

To install network tools for debugging:

  1. Check if telnet is installed by entering which telnet at the command prompt: 
bash-5.0# which telnet
bash-5.0#

📘

NOTE:

If the which telnet command does not return a path pointing to the telnet executable, then telnet is not available and you must install it.

  1. To update the Package Manager, enter apk update at the command prompt:
bash-5.0# apk update
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/main/x86_64/APKINDEX.tar.gz
fetch http://dl-cdn.alpinelinux.org/alpine/v3.11/community/x86_64/APKINDEX.tar.gz
v3.11.12-1-gede026da09 [http://dl-cdn.alpinelinux.org/alpine/v3.11/main]
v3.11.11-124-gf2729ece5a [http://dl-cdn.alpinelinux.org/alpine/v3.11/community]
OK: 11275 distinct packages available
  1. To install extra tools (including telnet), enter apk add busybox-extras at the command prompt:
bash-5.0# apk add busybox-extras
(1/1) Installing busybox-extras (1.31.1-r10)
Executing busybox-extras-1.31.1-r10.post-install
Executing busybox-1.31.1-r10.trigger
OK: 33 MiB in 36 packages
  1. To verify that telnet was installed, enter which telnet at the command prompt: 
bash-5.0# which telnet
/usr/bin/telnet
bash-5.0#
  1. To install additional network tools, enter apk add --update-cache iproute2 bind-tools net-tools at the command prompt.

Check Listening Ports

To check the listening ports, enter netstat -tuna at the command prompt:

netstat -tuna
bash-5.0# netstat -tuna
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State
tcp        0      0 0.0.0.0:8080            0.0.0.0:*               LISTEN
tcp        0      0 0.0.0.0:8443            0.0.0.0:*               LISTEN
tcp        0      0 172.x.x.x:xxxxx         199.x.x.x:80            TIME_WAIT
tcp        0      0 172.x.x.x:xxxxx         199.x.x.x:80            TIME_WAIT
tcp        0      0 172.x.x.x:xxxxx         199.x.x.x:80            TIME_WAIT

Check DNS Resolution

To check the DNS resolution, enter nslookup localhost at the command prompt:

bash-5.0# nslookup localhost
Server:		10.60.0.2
Address:	10.60.0.2#53
Non-authoritative answer:
Name:	localhost
Address: 127.0.0.1
Name:	localhost
Address: ::1

Check IP Addresses

To check the IP addresses, enter ip address at the command prompt:

ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
4: eth0@if5: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP group default
    link/ether 02:42:ac:11:00:02 brd ff:ff:ff:ff:ff:ff link-netnsid 0
    inet 172.x.x.x/16 brd 172.x.x.x scope global eth0
       valid_lft forever preferred_lft forever

Connect to Cloudwatch Logs Endpoint

To connect to the CloudWatch logs endpoint through telnet, enter telnet logs..amazonaws.com 443 at the command prompt:

telnet logs.<tdp-aws-region>.amazonaws.com 443
bash-5.0# telnet logs.eu-central-1.amazonaws.com 443
Connected to logs.eu-central-1.amazonaws.com

Connect to S3 through Telnet

To connect to S3 through telnet, enter telnet s3..amazonaws.com 443 at the command prompt:

telnet s3.<s3-aws-region>.amazonaws.com 443
bash-5.0# telnet s3.eu-central-1.amazonaws.com 443
Connected to s3.eu-central-1.amazonaws.com

Verify DNS Resolution to S3 Endpoints

To verify that your DNS can resolve to the S3 endpoints, enter nslookup s3.amazonaws.com at the command prompt. 

nslookup s3.amazonaws.com

View Container Environment Variables

To review environment variables and information used by the connector container at runtime, enter env at the command prompt.

env

Updated 8 months ago