A project to gather, explore, analyse and present train related data collected primarily from the Network Rail feeds. Currently, a graphical representation of the UK rail network is generated from raw train movement data and presented using a Dash application. The live dashboard for this project is located at thetrains.co.uk.
The full deployment consists of four containers:
- mongo - A MongoDB database instance to persist all data.
- collector - A process to robustly gather a large volume of live train movement data from the Network Rail feeds for storage in the MongoDB instance.
- generator - A data transformation process that runs on a timer. Its purpose is to clean and process the raw train movement data to generate a novel graphical representation of the U.K rail network. A Fruchterman-Reingold force-directed algorithm is used for the estimation of node positions.
- dash - A Dash frontend application to display the generated graphical network alongside live network usage.
The wiki contains information on the projects inner workings.
First, a .env file containing all the required environment variables is needed. See .env.example for an example. It is recommended to modify this file and rename it to .env. To build and run all the containers locally run:
make buildYou can then view the dash application at localhost:8000. To stop all the containers run:
make downFirst, the ./k8s/setup.yaml file needs to be modified for your cluster setup. Additionally, You will need both kubectl and skaffold for deployment.
To run the initial cluster setup run:
make k8s_setupThis step sets up the namespace, configuration, certificates, the application service and ingress. To build and push the containers to the GitHub container repository and then deploy to the cluster run:
make k8s_deployTo continuously build and deploy to the cluster while you make changes run:
skaffold devThe tests and other checking is run within a docker container, use by running:
make test