Developers documentation

This section provides documentation for developers/maintainers of the Baseline system. If you fall into this group, you should find all relevant information here.

System overview

The basic structure of the system is shown in the picture below:

overview

The main parts of the system are:

  1. Datastore server with Postgres database, usually referred to as backend
  2. Web application (originaly called oko, then DNEye, now Baseline, which is also used as a name for the whole system), usually called frontend
  3. Utility parts:
    • Logging server
    • Authentication and authorization server
    • Importing scripts

We will discuss these parts individually in the following sections.

Backend

The backend part of the system consists of relational database and the Datastore server exposing the main REST API. The server is implemented in ASP.NET 6.0. Source codes can be found here.

Object-database mapping

The object-database mapping is implemented using Entity Framework. We use the code first approach which means that the database model is derived by the framework based on the object structure in the code. The project is still under development, which means the database scheme is not constant and may change with time. Still we want to be able to push new updates of the Datastore server into production, where the production data is already present. Hence, we need to address the issue of how a new version of Datastore can operate on the database with older (i.e. non-compatible) scheme. We achieve this by running automated database migrations on Datastore start. This is done usnder the hood and no manual action is required.

Logging

We use Serilog logging framework.

Authentication and authorization

Datastore server implement JSON Web Tokens (JWT, https://jwt.io/) for authentication and authorization. Permissions of a specific user are given by the roles issues in the access token. See the Security page.

CI

Datastore project uses GitLab pipelines. The development always happens in a branch. Each push to the branch triggers a new baseline run with two steps - build and unit tests. The master branch is protected and cannot be pushed into. Once a branch pipeline is OK, a new merge request is created. The merge triggers the master pipeline containing three steps - build, unit tests and docker. Docker container is the only artifact produced by the pipeline.

Deployment

[todo]

DB

The backend uses a PostgreSQL DB. Deployment files can be found in the gitlab repo. Deployment of DB is part of the datastorenet deployment.

DB backup

The PostgreSQL DB is backed up by a k8s CronJob. Cronjob is started with kubectl apply -n kabanos-ns -f ./backup-job.yaml in the backup/ subdirectory of the postgres gitlab repo. The cronjob needs to have a pgpass secret. The content of the pgpass looks like this:

postgres-svc:5432:ngs:postgres:*password* // replace *password* with the actual password

Run base64 on the pgpass file to get the secret value and put it into the pgpass-secret.yaml file. Then run kubectl apply -n kabanos-ns -f ./pgpass-secret.yaml to create the secret. See this post for additional details.

The backups are stored in the DBapp project folder on the shared elixir storage. (Storage connected via PVC, see "create PVC as CIFS storage from CERIT-SC" part of the cerit documentation where the secret.yaml file is used.)

Restore from backup

Connect to the postgres pod through Rancher or kubectl exec ... (e.g kubectl exec -it postgres-5fb8fccc5-gkkmq -n kabanos-ns -- /bin/bash).

cd /mnt/backup
gunzip selected_backup.gz
psql -U postgres template1 -c 'drop database ngs;'
## the drop may fail with something like: ERROR:  database "ngs" is being accessed by other users
## ... just wait and try again later
psql -U postgres template1 -c 'create database ngs with owner postgres;'
psql -U postgres ngs < /mnt/backup/{selected_backup}

Swagger

Latest swagger document can be found here.

Frontend

Baseline frontend web application is written in ReactJS. Server-side of this application is provided by ExpressJS/Node stack in development mode and NGINX with Openresty in production mode.

URLs and routing

Frontend routing is implemented using react-router-dom library, using BrowserRouter in v6. As Baseline is a Single Page Application (SPA), the routing between URLs is handled completely in the browser, i.e. by client-side Javascript. This means that the server provides the whole application to the browser when the page is first loaded in the browser and all subsequent URL transitions are handled in the browser, meaning that the BrowserRouter renders and refreshes all relevant application parts depending on the URL being loaded. There is, however, a challenge if we want to implement link sharing. An example situation:

  1. User A is viewing patient ABC on URL https://baseline.dyn.cloud.trusted.e-infra.cz/patients/ABC and shares this link to User B for discussion
  2. User B opens the link in their browser.

However, there is no Baseline application code (javascript) loaded yet in User B's browser and no BrowserRouter to handle the URL. The action B in reality results in a request of /patients/ABC to the server (ExpressJS or NGINX). But the server is not handlind application-specific URLs -- in case of SPAs, the server only serves the whole application (main index.html). Due to this, such request would end with 404 Not found error from server and two confused Users (A and B). One of the ways of handling this issue is handling the / route on the server and providing the main application (index.html) when this URL is requested. This is implemented in both development (ExpressJS) and Production (NGINX config) scenarios.