Addendum: Docker Overview

A brief guide to Docker for better understanding how POCS works.

Docker for PANOPTES

This section is not necessary for running POCS or the software, but may be helpful for a deeper understanding of what is going on.

There are many good guides to Docker on the internet, and the official documentation is quite good, so this page just presents a quick overview of Docker in the context of POCS and how the system is run.

Docker itself acts sort of like other virtualization software such as VirtualBox or VMWare, but has a different design and different purposes.

In simple terms, the difference would be that while normal virtualization software is designed to mimic an entire operating system, Docker should be thought of as providing minimal "services," or what are known as "microservices."

For our purposes, it's enough to know a few key terms and understand the basic actions a user can perform to start and stop these "microservices".

Microservices

Ideal Docker services are implemented as a network of microservices that interact with each other. For instance, one could start up a config-server, a database-server, and a web-server. Each of these perform a slightly different service for a larger application. In this scenario, one could make it so that only the web-server was exposed to the public, minimizing the risk someone could gain access to your database or other services.

The way Docker is used for panoptes-utils and panoptes-pocs services is not "micro" in any sense of the word. The images (see below) are large and bloated and the services are not designed to be quickly stopped and restarted like would be done with Docker best practices.

Instead, PANOPTES uses Docker as a convenient way to pre-install everything that is needed to run a PANOPTES unit. In this sense it is much more like a regular virtualization software that it should be.

Despite being large, the rest of the serivces and how they are used is the same as if they were micro. 🤷

Terminology

Term

Definition

image

The "virtual machine" that has been built with all required dependencies installed. The dependencies and other install instructions are defined in a Dockerfile.

An image usually has a simple name that relates to the service it provides, e.g. config-server.

Images can also have multiple tags.

tag

A simple label for an image that lets the user select from different types of the same image. The default tag is latest. Other common tags are develop as well as those corresponding to specific version numbers of the service. Tags can also be used to distinguish between computer architectures (e.g. for a Raspberry Pi - arm64).

container

A running copy of a tagged image. There can be many running containers from a single tagged image. Each container is running its own "copy" of the service and is by default isolated from other containers although they can communicate with each other via a shared private network, a shared directory, or various other means.

service

A logical concept about what it is the a Docker container should be doing, e.g. a web-server.

Dockerfile

The plain text file that defines the commands used to create the image as well as the commands that are run when a container is started.

registry

An online service that hosts images for downloading. The default registry for Docker is https://hub.docker.io.

PANOPTES uses Google Cloud Registry, which provides images at gcr.io.

Basic Docker workflow

The basic workflow for Docker has two components: developers who provide the built and tagged images to a registry, and users who download (i.e. pull) images from a registry where they can then be run as containers.

For Developers of new PANOPTES services

Developers primarily need to write three files: a definition file, a build file, and a run file:

  1. Defining Images: Write a Dockerfile that defines everything necessary for building an image as well as providing a default run command.

  2. Building Images: Write a cloudbuild.yaml file that defines how to automatically build the image from the Dockerfile using Google Cloud Build (GCB).

  3. Running Images: Write a docker-compose.yaml file that defines how to run one or more containers based off the images.

There are also "triggers" that need to be defined on the Google Cloud side of things, but that's more detail than this guide provides. These are a one-time setup and should be done for all relevant repositories.

For Users of PANOPTES services

Users in this case could either be builders of PANOPTES units or developers of other PANOPTES software.

The pull and run commands below are actually hidden from normal PANOPTES users. The pull happens during the scripted install and the run is wrapped by our pocs up command. See Developer Install for more info.

Users typically need to do two things, namely pull and run images.

  1. Pull Image: Use the docker pull command to pull an image from a registry. The full name for the image, including host and tag, needs to be included otherwise they will default to hub.docker.io and latest respectively.

  2. Run Services: Use docker run to create and run containers based on the images.