Evaluate alternative VM/container solutions

[rmcewan]

Longer term, the VM should be positioned for easy adoption by less technical players, e.g., NLP researchers without deep IT experience. Put together a list of alternative VM/container solutions with their pros/cons for us to evaluate.

This should not start before the until the Initial Plumbing milestone is complete.

[GregSilverman]

Can we assume these NLP researchers "without deep IT experience" at least are capable of running a script from the command line?

[rmcewan]

Yes, I believe we are assuming a research is a data science or programming-familiar professional. Should have cmd line experience.

[GregSilverman]

I think Ansible with Docker would fit this requirement. I've played around with Ansible and Vagrant for VM management, so I believe this may be a good direction to pursue. Will delve into this and compare with other similar solutions in due course (as per completion of priority involving Initial Plumbing Milestone).

[GregSilverman]

Installed Docker CE. Will start scripting creation of some simple container apps. So far this appears to be very resource non-intensive (unlike use of a VM).

[GregSilverman]

Downside to use of Docker Containers: Docker containers are shared in Docker container registries. See exhaustive detail here - sharing Docker container images. The Docker registry is unfortunately public (no options for private).

Two candidate options are:

1. It is possible to easily set up a private repository on a local server.

2. Another option is to write a set of scripts with Ansible playbooks to create and automate the creation of a Docker container image. The downside to this is that installation of MetaMap is done via authentication to the UMLS web site (with cookie authentication and CAS - ironically, central authentication service - that issues a unique id for the session - see script to load CAS protected content using curl ...) and would take a bit to hack a curl command to grab the files.

In any case, both above options are far preferable than dealing with a VM image. Download time for the image for me - on campus - was 55 minutes. Also the VM image is both a space and resource hog.

[GregSilverman]

I am going to build a simple Docker image that has cTAKES in it (since it only uses UMLS credentials and not the entire UMLS directory structure). I will first build it as a standard Docker image from the command line and then I will use the following method recommended by the Ansible project:

1. Build base Docker image with Ansible installed
2. Create playbook to install and configure cTAKES
3. Invoke Ansible

This should give us some good data for any downstream decisions we need to make.

NB: there are other container solutions, most notably Red Hat's MiniShift, but that run a cluster inside a local VM. Not sure if this is the direction we would want to go, especially since Docker can run container clusters across multiple machines. Just something to keep in mind...

[GregSilverman]

Proof of concept in the works to get cTAKES CPE GUI running in its own container via x11.

[GregSilverman]

Got an x11 container working using CPE GUI:

docker run -d --name ctakes -e DISPLAY=$ip:0 -v ~/development/ansible/playbooks/test/data:/ansible/test/data -v /tmp/.X11-unix:/tmp/.X11-unix gms/ctakes

Decided to scrap due to various issues related to interface. Now am able to run as command line with logging to console output.

Assuming that Docker image has been built, steps to get logged to console are:

# if container exists then delete it
if [ "$(docker ps -aq -f status=exited -f name=ctakes)" ]; then
    docker rm ctakes
fi
# run container
docker run -d   -v ~/development/ansible/playbooks/test/data:/ansible/test/data gms/ctakes

# output logs to console
docker logs ctakes --follow

#check container status (exit status > 0 means failure)
docker container ls -a

[GregSilverman]

Using x11 Dockerfile, I will work next on getting NLP-TAB containerized.

[GregSilverman]

I am now looking into using ansible_container. It is designed to orchestrate building multiple dependency containers utilizing multiple services. It looks much easier than the straight Docker way (using Docker Compose).