Skip to content

MongoDB setup

Jump to bottom
Panagiotis Moulos edited this page Jan 30, 2026 · 5 revisions

Setup MongoDB

The platform and the annotation system will have MongoDB as the backend storage system. This section describes installation and customization of MongoDB in the backend server. We mostly follow the official instructions.

Number of open files and processes

MongoDB requires a minimum number of allowed opened files and processes to guarantee good functionality. These are either controlled temporarily by the ulimit command or permanently by modifying the /etc/security/limits.conf file.

To work with the installation, the following is enough:

ulimit -n 65536

To make the required changes permanent, enter (as root) the following lines in /etc/security/limits.conf (before # End of file):

* soft nofile 65536
* hard nofile 65536
* soft memlock unlimited
* hard memlock unlimited

MongoDB installation

We then follow steps 1-4 here. After completion and prior to starting the MonfoDB service, we need to configure where the MongoDB files will be stored. As the expected data will be of great volumes, we must ensure that they will live in a filesystem with sufficient space. In the following, it is assumed as $DBHOME. Other dynamic variables which are commited here only as examples are the database ports.

MongoDB storage

We assume that:

  • The storage volume with adequate storage is in /media/big_storage
  • User (or main app database) port is 27017
  • Bacground knowledge database port is 27018 
DBHOME=/media/big_storage/mongodb
USERDB_PORT=27017
BACKDB_PORT=27018

sudo mkdir -p $DBHOME
sudo chown -R mongodb:mongodb $DBHOME

Create separate spaces for user database and background knowledge database as well as the configurations:

sudo mkdir -p $DBHOME/userdb && sudo chown -R mongodb:mongodb $DBHOME/userdb
sudo mkdir -p $DBHOME/backdb && sudo chown -R mongodb:mongodb $DBHOME/backdb
sudo mkdir -p $DBHOME/config && sudo chown -R mongodb:mongodb $DBHOME/config
sudo mkdir -p $DBHOME/logs && sudo chown -R mongodb:mongodb $DBHOME/logs

We then proceed to create root users and mongod configuration files.

Root users

Prior to enabling security in MongoDB we create root users for each database:

sudo /usr/bin/mongod \
  --port $USERDB_PORT \
  --dbpath $DBHOME/userdb \
  --pidfilepath /tmp/user.pid \
  --logpath /tmp/user.log \
  --noauth --quiet &

Then create the main root user in the MongoDB shell:

sudo /usr/bin/mongosh --port $USERDB_PORT --authenticationDatabase admin

followed by:

use admin
db.createUser({
  user: "root",
  pwd: passwordPrompt(),
  roles: ["root"]
})
exit

And stop the instance:

sudo /usr/bin/mongod \
  --port $USERDB_PORT \
  --dbpath $DBHOME/userdb \
  --pidfilepath /tmp/user.pid \
  --shutdown

Repeat for the background database by replacing user with back in --dbpath, --pidfilepath and --logpath as well as the port.

mongod config files

The data storage model we follow consists of two instances:

  • An instance to hold main application data (user data, analyses, VCF file contents, annotations etc.)
  • An instance to store the background annotation knowledge gathered from public resources

For the first instance (user data) the file will be stored in $DBHOME/config/mongod_userdb.conf. To create it, use the following:

echo -e \
"# mongod_userdb.conf

storage:
  dbPath: $DBHOME/userdb

systemLog:
  destination: file
  logAppend: true
  path: $DBHOME/logs/mongod_userdb.log
  logRotate: rename
  timeStampFormat: iso8601-local

net:
  port: $USERDB_PORT
  bindIp: 127.0.0.1 # More addresses will be added

processManagement:
  pidFilePath: $DBHOME/config/mongod_userdb.pid

security:
  authorization: enabled

#operationProfiling:

#replication:

#sharding:
" | sudo tee $DBHOME/config/mongod_userdb.conf && \
sudo chown mongodb:mongodb $DBHOME/config/mongod_userdb.conf

For the second instance (background data) the file will be stored in $DBHOME/config/mongod_backdb.conf:

echo -e \
"# mongod_backdb.conf

storage:
  dbPath: $DBHOME/backdb

systemLog:
  destination: file
  logAppend: true
  path: $DBHOME/logs/mongod_backdb.log
  logRotate: rename
  timeStampFormat: iso8601-local

net:
  port: $BACKDB_PORT
  bindIp: 127.0.0.1 # More addresses will be added

processManagement:
  pidFilePath: $DBHOME/config/mongod_backdb.pid

security:
  authorization: enabled

#operationProfiling:

#replication:

#sharding:
" | sudo tee $DBHOME/config/mongod_backdb.conf && \
sudo chown mongodb:mongodb $DBHOME/config/mongod_backdb.conf

Start the daemons

Firstly start the daemons:

sudo /usr/bin/mongod --config $DBHOME/config/mongod_userdb.conf &
sudo /usr/bin/mongod --config $DBHOME/config/mongod_backdb.conf &

And then change the ownerships of the log and pid files so that mongodb user can take over:

sudo chown mongodb:mongodb $DBHOME/logs/mongod_userdb.log
sudo chown mongodb:mongodb $DBHOME/logs/mongod_backdb.log
sudo chown mongodb:mongodb $DBHOME/config/mongod_userdb.pid
sudo chown mongodb:mongodb $DBHOME/config/mongod_backdb.pid

Stop the daemons

To stop the daemons:

sudo /usr/bin/mongod --shutdown --config $DBHOME/config/mongod_userdb.conf
sudo /usr/bin/mongod --shutdown --config $DBHOME/config/mongod_backdb.conf

Initialization of default users

We need three users:

  • A global user to read/write in both static and user databases
  • A user to read/write in the user production database
  • A user to read/write in the user test database

Instructions to create these users are placed in a non-commited file. The main MongoDB commands are something like the following:

Firstly open the MongoDB shell:

sudo /usr/bin/mongosh --host HOST --port USERDB_PORT \
  --username ADMIN_ROOT_USER --password ADMIN_ROOT_USER \
  --authenticationDatabase admin

And then create the users:

use dbname
db.createUser({
    user: "UserName",
    pwd: "StrongPassword",
    roles: [{ 
        role: "readWrite", 
        db: "dbname" 
    },{ 
        role: "readWrite",
        db: "anotherdbname" 
    },{ 
        role: "read", 
        db: "readonlydbname" 
    }]
})

Instances as system services

To avoid manual restart of MongoDB and better system integration while in production level, it is recommened to configure the user and backend database instances as system services (systemd). The instructions below outline this process.

Create service files

For user database:

sudo nano /etc/systemd/system/mongodb-userdb.service

And then, paste the following:

[Unit]
Description=MongoDB Database Server (UserDB)
Documentation=https://docs.mongodb.org/manual
After=network.target

[Service]
Type=simple
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --config $DBHOME/config/mongod_userdb.conf
ExecStop=/usr/bin/mongod --config $DBHOME/config/mongod_userdb.conf --shutdown
PIDFile=$DBHOME/config/mongod_userdb.pid

# Security
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=$DBHOME

# Resource management
LimitFSIZE=infinity
LimitCPU=infinity
LimitAS=infinity
LimitNOFILE=64000
LimitNPROC=64000

# Restart settings
Restart=always
RestartSec=10
StartLimitInterval=90
StartLimitBurst=10

[Install]
WantedBy=multi-user.target

For backend database:

sudo nano /etc/systemd/system/mongodb-backdb.service

And then, paste the following:

[Unit]
Description=MongoDB Database Server (BackDB)
Documentation=https://docs.mongodb.org/manual
After=network.target

[Service]
Type=simple
User=mongodb
Group=mongodb
ExecStart=/usr/bin/mongod --config $DBHOME/config/mongod_backdb.conf
ExecStop=/usr/bin/mongod --config $DBHOME/config/mongod_backdb.conf --shutdown
PIDFile=$DBHOME/config/mongod_backdb.pid

# Security
PrivateTmp=true
ProtectSystem=full
ReadWritePaths=$DBHOME

# Resource management
LimitFSIZE=infinity
LimitCPU=infinity
LimitAS=infinity
LimitNOFILE=64000
LimitNPROC=64000

# Restart settings
Restart=always
RestartSec=10
StartLimitInterval=90
StartLimitBurst=10

[Install]
WantedBy=multi-user.target

Manage forking in config files

Edit each of the $DBHOME/config/mongod_userdb.conf and $DBHOME/config/mongod_backdb.conf and add fork: false under processManagement:

processManagement:
  ...
  fork: false

Reload systemd and start services

First kill any MongoDB instances running, otherwise we will have conflicts:

sudo /usr/bin/mongod --config $DBHOME/config/mongod_userdb.conf --shutdown
sudo /usr/bin/mongod --config $DBHOME/config/mongod_backdb.conf --shutdown

change the ownership of database files (owned by root if started manually):

sudo chown -R mongodb:mongodb $DBHOME/userdb
sudo chown -R mongodb:mongodb $DBHOME/backdb

and restart as system services:

# Reload systemd to recognize new services
sudo systemctl daemon-reload

# Start the services
sudo systemctl start mongodb-userdb
sudo systemctl start mongodb-backdb

# Enable services to start on boot
sudo systemctl enable mongodb-userdb
sudo systemctl enable mongodb-backdb

Local resources

MongoDB setup

Frontend setup

Clone this wiki locally