[WIP] Implement deploy example using Kamal

README.md

@@ -41,7 +41,9 @@ MONEYFORWARD_PASSWORD=Passw0rd!
 YNAB_ACCESS_TOKEN=abunchofcharacters
 ```
 
-Alternatively, you can also use something [1Password's CLI](https://developer.1password.com/docs/cli/)
+<a name="one-password-cli"></a>
+
+Alternatively, you can also use something like [1Password's CLI](https://developer.1password.com/docs/cli/)
 to avoid storing clear secrets:
 
 ```
@@ -71,6 +73,44 @@ op run --env-file=.env -- mfynab mfynab-david.yml
 After checking out the repo, run `bundle install` to install dependencies.
 Then, run `bin/rake test` to run the tests.
 
+## Running MFYNAB with cron and Docker
+
+The `docker-example/` directory contains sample files that'll help you schedule MFYNAB inside a Docker container.
+
+See the comments in each file for more details on how it works.
+
+First you'll want to bring your MFYNAB configuration file into this directory:
+
+```sh
+cp path_to/config.yml docker_example/
+```
+
+Then you can build the Docker image:
+
+```sh
+docker build -t mfynab docker_example/
+```
+
+Finally, you can run the Docker image. Note that you need to pass secrets as environment variables:
+
+```sh
+docker run -d \
+  --env YNAB_ACCESS_TOKEN=... \
+  --env MONEYFORWARD_USERNAME=... \
+  --env MONEYFORWARD_PASSWORD='...' \
+  --name mfynab mfynab
+```
+
+You can also use the 1Password CLI ([documented here](#one-password-cli)) for this step:
+
+```sh
+op run --env-file=.env -- sh -c 'docker run -d \
+  --env YNAB_ACCESS_TOKEN=$YNAB_ACCESS_TOKEN \
+  --env MONEYFORWARD_USERNAME=$MONEYFORWARD_USERNAME \
+  --env MONEYFORWARD_PASSWORD="$MONEYFORWARD_PASSWORD" \
+  --name mfynab mfynab'
+```
+
 ## Roadmap
 
 ### Deploy/Automate
@@ -115,3 +155,10 @@ Previous notes:
   ```
 - Passing logger everywhere feels weird.
 - Prompt user for captcha and other account extra authentication required by Money Forward?
+- One might want to run a single Docker instance for multiple users, but the current setup does not allow that easily. We'll want to bring the secret environment variables into the config file, making it possible to assign them to a given "user", and name them accordingly.
+
+
+
+WIP:
+Container on Kamal fails to log in because of the "Additional Authentication via Email".
+Will have to implement the "log in on deploy" to get it to work.

docker_example/.gitignore

@@ -0,0 +1 @@
+config.yml

docker_example/Dockerfile

@@ -0,0 +1,22 @@
+FROM ruby:3.4
+
+# Install cron and chromium
+RUN apt-get update && apt-get install -y cron chromium
+
+WORKDIR /app
+
+# Copy Gemfile first and install gems
+COPY Gemfile Gemfile.lock ./
+RUN bundle install
+
+# Copy the rest of the application (this two-step process helps with layer caching)
+COPY . .
+
+# Add cron job
+RUN crontab /app/mfynab-cron
+
+# Add a health check to ensure the cron process is running
+HEALTHCHECK --interval=30s --timeout=10s --start-period=5s --retries=3 CMD pgrep cron
+
+# Run the entrypoint script by default
+CMD [ "/app/entrypoint.sh" ]

docker_example/Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+# FIXME: point to the latest RubyGems release
+gem "mfynab", github: "davidstosik/mfynab", branch: "sto/docker"

docker_example/Gemfile.lock

@@ -0,0 +1,62 @@
+GIT
+  remote: https://github.com/davidstosik/mfynab.git
+  revision: 73f36f266bbd879983644c8aadc2189b70d22665
+  branch: sto/docker
+  specs:
+    mfynab (0.1.4)
+      csv
+      ferrum (~> 0.15)
+      nokogiri (~> 1.18)
+      psych
+      ynab (~> 3.6)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    addressable (2.8.7)
+      public_suffix (>= 2.0.2, < 7.0)
+    base64 (0.2.0)
+    concurrent-ruby (1.3.5)
+    csv (3.3.2)
+    date (3.4.1)
+    ethon (0.16.0)
+      ffi (>= 1.15.0)
+    ferrum (0.16)
+      addressable (~> 2.5)
+      base64 (~> 0.2)
+      concurrent-ruby (~> 1.1)
+      webrick (~> 1.7)
+      websocket-driver (~> 0.7)
+    ffi (1.17.1)
+    ffi (1.17.1-arm64-darwin)
+    mini_portile2 (2.8.8)
+    nokogiri (1.18.3)
+      mini_portile2 (~> 2.8.2)
+      racc (~> 1.4)
+    nokogiri (1.18.3-arm64-darwin)
+      racc (~> 1.4)
+    psych (5.2.3)
+      date
+      stringio
+    public_suffix (6.0.1)
+    racc (1.8.1)
+    stringio (3.1.5)
+    typhoeus (1.4.1)
+      ethon (>= 0.9.0)
+    webrick (1.9.1)
+    websocket-driver (0.7.7)
+      base64
+      websocket-extensions (>= 0.1.0)
+    websocket-extensions (0.1.5)
+    ynab (3.6.0)
+      typhoeus (~> 1.0, >= 1.0.1)
+
+PLATFORMS
+  arm64-darwin-24
+  ruby
+
+DEPENDENCIES
+  mfynab!
+
+BUNDLED WITH
+   2.6.2

docker_example/entrypoint.sh

@@ -0,0 +1,7 @@
+#!/usr/bin/env sh
+
+# Save environment variables to a file, so we can forward them to cron jobs
+env >> /etc/environment
+
+# Run cron in the foreground
+cron -f

docker_example/kamal/.kamal/secrets

@@ -0,0 +1,8 @@
+SECRETS=$(bundle exec kamal secrets fetch --adapter 1password --account my 'Private/GitHub/secrets/registry token' 'Private/Moneyforward/username' 'Private/Moneyforward' 'Private/YNAB/secrets/API token')
+
+GHCR_REGISTRY_PASSWORD=$(kamal secrets extract 'Private/GitHub/secrets/registry token' $SECRETS)
+MONEYFORWARD_USERNAME=$(kamal secrets extract 'Private/Moneyforward/username' $SECRETS)
+MONEYFORWARD_PASSWORD=$(kamal secrets extract 'Private/Moneyforward' $SECRETS)
+YNAB_ACCESS_TOKEN=$(kamal secrets extract 'Private/YNAB/secrets/API token' $SECRETS)
+# FIXME: test cookie validity, renew if necessary, worst case open a browser to let the user log in (eg, `mfynab login`?)
+COOKIE_CACHE_PRIME=$(cat ~/.config/mfynab/cookie)

docker_example/kamal/Dockerfile

@@ -0,0 +1,22 @@
+FROM ruby:3.4
+
+# Install cron and chromium
+RUN apt-get update && apt-get install -y cron chromium
+
+WORKDIR /app
+
+# Copy Gemfile first and install gems
+COPY Gemfile Gemfile.lock ./
+RUN bundle install
+
+# Copy the rest of the application (this two-step process helps with layer caching)
+COPY . .
+
+# Add cron job
+RUN crontab /app/config/crontab
+
+# Add a health check to ensure the cron process is running
+HEALTHCHECK --interval=30s --timeout=10s --start-period=60s --retries=3 CMD pgrep cron
+
+# Run the entrypoint script by default
+CMD [ "/app/entrypoint.sh" ]

docker_example/kamal/Gemfile

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+source "https://rubygems.org"
+
+gem "kamal", "~> 2.5"
+gem "mfynab", github: "davidstosik/mfynab", branch: "sto/kamal"

docker_example/kamal/Gemfile.lock

@@ -0,0 +1,145 @@
+GIT
+  remote: https://github.com/davidstosik/mfynab.git
+  revision: 764740ee3b6d5f5f3b9a84924134c65e08c5e383
+  branch: sto/kamal
+  specs:
+    mfynab (0.1.4)
+      csv
+      ferrum (~> 0.15)
+      nokogiri (~> 1.18)
+      psych
+      ynab (~> 3.6)
+
+GEM
+  remote: https://rubygems.org/
+  specs:
+    activesupport (8.0.1)
+      base64
+      benchmark (>= 0.3)
+      bigdecimal
+      concurrent-ruby (~> 1.0, >= 1.3.1)
+      connection_pool (>= 2.2.5)
+      drb
+      i18n (>= 1.6, < 2)
+      logger (>= 1.4.2)
+      minitest (>= 5.1)
+      securerandom (>= 0.3)
+      tzinfo (~> 2.0, >= 2.0.5)
+      uri (>= 0.13.1)
+    addressable (2.8.7)
+      public_suffix (>= 2.0.2, < 7.0)
+    base64 (0.2.0)
+    bcrypt_pbkdf (1.1.1)
+    benchmark (0.4.0)
+    bigdecimal (3.1.9)
+    concurrent-ruby (1.3.5)
+    connection_pool (2.5.0)
+    csv (3.3.2)
+    date (3.4.1)
+    dotenv (3.1.7)
+    drb (2.2.1)
+    ed25519 (1.3.0)
+    ethon (0.16.0)
+      ffi (>= 1.15.0)
+    ferrum (0.16)
+      addressable (~> 2.5)
+      base64 (~> 0.2)
+      concurrent-ruby (~> 1.1)
+      webrick (~> 1.7)
+      websocket-driver (~> 0.7)
+    ffi (1.17.1)
+    ffi (1.17.1-aarch64-linux-gnu)
+    ffi (1.17.1-aarch64-linux-musl)
+    ffi (1.17.1-arm-linux-gnu)
+    ffi (1.17.1-arm-linux-musl)
+    ffi (1.17.1-arm64-darwin)
+    ffi (1.17.1-x86-linux-gnu)
+    ffi (1.17.1-x86-linux-musl)
+    ffi (1.17.1-x86_64-linux-gnu)
+    ffi (1.17.1-x86_64-linux-musl)
+    i18n (1.14.7)
+      concurrent-ruby (~> 1.0)
+    kamal (2.5.3)
+      activesupport (>= 7.0)
+      base64 (~> 0.2)
+      bcrypt_pbkdf (~> 1.0)
+      concurrent-ruby (~> 1.2)
+      dotenv (~> 3.1)
+      ed25519 (~> 1.2)
+      net-ssh (~> 7.3)
+      sshkit (>= 1.23.0, < 2.0)
+      thor (~> 1.3)
+      zeitwerk (>= 2.6.18, < 3.0)
+    logger (1.6.6)
+    mini_portile2 (2.8.8)
+    minitest (5.25.4)
+    net-scp (4.1.0)
+      net-ssh (>= 2.6.5, < 8.0.0)
+    net-sftp (4.0.0)
+      net-ssh (>= 5.0.0, < 8.0.0)
+    net-ssh (7.3.0)
+    nokogiri (1.18.3)
+      mini_portile2 (~> 2.8.2)
+      racc (~> 1.4)
+    nokogiri (1.18.3-aarch64-linux-gnu)
+      racc (~> 1.4)
+    nokogiri (1.18.3-aarch64-linux-musl)
+      racc (~> 1.4)
+    nokogiri (1.18.3-arm-linux-gnu)
+      racc (~> 1.4)
+    nokogiri (1.18.3-arm-linux-musl)
+      racc (~> 1.4)
+    nokogiri (1.18.3-arm64-darwin)
+      racc (~> 1.4)
+    nokogiri (1.18.3-x86_64-linux-gnu)
+      racc (~> 1.4)
+    nokogiri (1.18.3-x86_64-linux-musl)
+      racc (~> 1.4)
+    ostruct (0.6.1)
+    psych (5.2.3)
+      date
+      stringio
+    public_suffix (6.0.1)
+    racc (1.8.1)
+    securerandom (0.4.1)
+    sshkit (1.24.0)
+      base64
+      logger
+      net-scp (>= 1.1.2)
+      net-sftp (>= 2.1.2)
+      net-ssh (>= 2.8.0)
+      ostruct
+    stringio (3.1.5)
+    thor (1.3.2)
+    typhoeus (1.4.1)
+      ethon (>= 0.9.0)
+    tzinfo (2.0.6)
+      concurrent-ruby (~> 1.0)
+    uri (1.0.3)
+    webrick (1.9.1)
+    websocket-driver (0.7.7)
+      base64
+      websocket-extensions (>= 0.1.0)
+    websocket-extensions (0.1.5)
+    ynab (3.6.0)
+      typhoeus (~> 1.0, >= 1.0.1)
+    zeitwerk (2.7.2)
+
+PLATFORMS
+  aarch64-linux-gnu
+  aarch64-linux-musl
+  arm-linux-gnu
+  arm-linux-musl
+  arm64-darwin-24
+  ruby
+  x86-linux-gnu
+  x86-linux-musl
+  x86_64-linux-gnu
+  x86_64-linux-musl
+
+DEPENDENCIES
+  kamal (~> 2.5)
+  mfynab!
+
+BUNDLED WITH
+   2.6.2

docker_example/kamal/config/crontab

@@ -0,0 +1,14 @@
+# Pick up environment variables that were forwarded when starting the container
+SHELL=/bin/bash
+BASH_ENV=/etc/environment
+
+# See:
+# https://www.reddit.com/r/docker/comments/mzqibu/how_to_correctly_run_cron_in_docker/
+
+# FIXME: I'd like to be able to use UTC+9 times
+# TODO: rethink schedule. From a bank perspective, 6pm and 6am might yield the same results (no activity overnight?)
+# Run MFYNAB at 6am, 12pm, and 6pm JST every day
+#* 3,9,21 * * * cd /app; bundle exec mfynab config/mfynab.yml >/proc/1/fd/1 2>/proc/1/fd/2
+
+# 9, 12, 15, 18, 21 UTC+9
+0 0,3,6,9,12 * * * cd /app; bundle exec mfynab config/mfynab.yml >/proc/1/fd/1 2>/proc/1/fd/2

docker_example/kamal/config/deploy.yml

@@ -0,0 +1,27 @@
+service: mfynab
+image: davidstosik/mfynab
+
+primary_role: cron
+servers:
+  cron:
+    hosts:
+      - 157.180.23.52
+    proxy: false
+
+registry:
+  server: ghcr.io
+  username: davidstosik
+  password:
+    - GHCR_REGISTRY_PASSWORD
+builder:
+  arch: amd64
+
+env:
+  secret:
+    - MONEYFORWARD_USERNAME
+    - MONEYFORWARD_PASSWORD
+    - YNAB_ACCESS_TOKEN
+    - COOKIE_CACHE_PRIME
+
+aliases:
+  shell: app exec --interactive --reuse "bash"