CompBio app server

This project uses:

• Docker Compose to manage multiple Docker containers
• Nginx as a reverse proxy (i.e. to redirect URLs), to serve SSL certificates and to show a public folder
• ShinyProxy to run R/Shiny and Python apps via Docker
• Celery to run background tasks via a job queue system
  • Flower to monitor Celery and to send jobs to Celery via its REST API
  • Redis to serve as message broker for celery
• Prometheus and Grafana to log and visualise metrics from ShinyProxy, Celery, Nginx and the system
  • Node exporter to monitor system resource usage
  • Nginx Prometheus exporter to monitor Nginx
• Plausible analytics to gather traffic metrics of multiple websites
  • Postgres database (Plausible's user data)
  • Clickhouse database (Plausible's analytics)
• RStudio Server to test code on-the-fly (dev profile only)

Setup

First, download or clone this repository to your personal computer.

To setup and start running the app server, you will need to build local Docker images and download Docker images used in Docker Compose and as ShinyProxy apps. This can be done in slightly different ways as detailed below.

Testing environment (i.e. staging)

To avoid testing changes in the production server, prepare a staging environment in your computer by going to the project folder and running:

./setup-testing-mode.sh            # setup files for testing and download Docker images
docker compose --profile dev up -d # start server in the `dev` profile and detached mode

You can now visit http://localhost in your web browser. The services should be fully operational in about ~30 seconds. Specific services may only be accessible via their port, e.g. http://localhost:8000 for plausible and http://localhost:8787 for RStudio.

Note

Some services are only available in the dev profile (RStudio).

Production environment

Some services are given a default email, user and/or password. The default ones are fine for testing purposes, but should be set in a .env file when running in a production environment.

Example `.env` file

RSTUDIO_PASSWORD=rstudio_pass

POSTGRES_USER=postgres_user
POSTGRES_PASSWORD=postgres_pass

GRAPHANA_USER=graphana_user
GRAPHANA_PASSWORD=graphana_pass

[email protected]
PLAUSIBLE_USER=plausible_user
PLAUSIBLE_PASSWORD=plausible_pass

After creating such a file, go to the project folder and run:

./setup.sh              # prepare config files and download Docker images
docker compose up -d    # start services in detached mode

You can now visit http://localhost in your web browser.

Tip

If you are using a remote machine, port forwarding allows to access specific services in your browser. For instance, to access http://localhost:8000 (plausible) in your browser, you first need to connect to the machine like so:

ssh -L 8000:localhost:8000 [server]

Next steps

• Add and update apps in ShinyProxy
• Monitor website analytics with Plausible
• Renew SSL certificates and other Nginx configurations

Essential Docker Compose commands

docker compose up -d --build      # build images and start in detached mode
docker compose down               # stop all services
docker compose restart shinyproxy # restart a specific service, e.g. shinyproxy
docker compose logs nginx         # print logs of a specific servcie, e.g. nginx
docker compose -h                 # show help documentation

Some of the services were configured to store their data in Docker volumes:

docker volume ls                  # list Docker volumes
docker volume inspect [volume]    # inspect volume information

Relevant assets

Asset	Description
docker-compose.yml	Docker Compose configuration
nginx	Nginx configuration files (main one is nginx.conf)
nginx/public	Publicly available data downloadable at /public
shinyproxy/application.yml	ShinyProxy configuration (including Shiny apps)
shinyproxy/templates	ShinyProxy custom HTML files
celery/tasks.py	Celery tasks

Sources of inspiration

• shrektan/shinyproxy-docker-compose-example
• kassambara/shinyproxy-config