API: Accept money and send postcards

[conorgil]

Goal: After successfully charing the user's CC for $X, send postcards to Y randomly chosen registered voters who have not previously received a postcard, where Y is the number of postcards possible to purchase for $X (minus all service fees).

If we embed Stripe into our site directly:

1. API request includes Stripe token. Charge the user's CC. Wait for successful response from Stripe.

If we use Square or something and they handle charging the user's CC on Stripe:

1. API request is a webhook from Stripe confirming that the CC was charged.

After payment is resolved:

2. Calculate how many postcards can be purchased after subtracting the Stripe service fees. Let that by Y.
3. Choose Y voters at random from the database who have not received a post card previously.
4. Make request to ClickSend API to send postcard

If we embed Stripe into our site directly:

5. Send response to our website so that we can show the user a confirmation message that the postcards were sent.

[conorgil]

@ravenac95 Payam, to get unblocked, you can create a ClickSend account with your own email. They give you a few dollars in credit so that you can play around with the API.

Once we get the non-profit legal entity set up, I will create new accounts in the name of the non-profit, setup billing, etc.

[ravenac95]

Some semi-organized notes for later...

Clicksend has a very straight-forward API. Based on my knowledge of stripe the following are the current thoughts on how we do this.

Required Infrastructure

• Github Actions / Some other simple worker
• DynamoDB (or something similarly simple)

Process

1. Once donor completes the transaction hopefully squarespace or what not will email the user directly on our behalf. (needs to be derisked)
2. Once every 5 mins a github action cron will run that will handle all of mail related work. The following is a bit of pythonic pseudocode for this process (the actual fields are not correct to each api...):

  # Download all of the latest `charge.succeeded` events from Stripe
  stripe_events = stripe.events.download_all()
  
  for event in stripe_events:
      # filter for charge.succeeded events
      if event.type != "charge.succeeded":
          continue
      # calculate how many postcards can be sent for a given charge
      charge_info = event.info
      number_of_postcards_to_send = charge_info.amount / POSTCARD_PRICE_CONSTANT

      # Randomly choose from the Voter database... 
      # let's assume for pseudo-code that it only chooses users who've never been selected
      random_voters = voter_db.choose_uncontacted_voters(number_of_postcards_to_send)
      
      # Call clicksend api to send voters their postcards
      for voter in random_voters:
          click_send.send_postcard(voter.address, ... other config values ...)

          # Update db so we don't double send to voters 
          # we assume this used by the voter_db above... ya this isn't super clean pseudo-code but hopefully it makes sense
          sent_addresses_db.insert(sha256(voter.name))

      # Send email to donor
      send_thank_you_email(
          event.customer.email, 
          "Thanks for your donation! You've sent a postcard to %d voter(s)" % len(random_voters),
      )

One other thing we could do is take any of the left over donations and send additional postcards. The postcards will cost us $0.65 for standard mail of $0.98 for priority mail.

Logging

If we choose to use github actions for this we'll need to ensure that no personal information is logged. If we deem this too much of a potential risk we should run this worker process elsewhere. Heroku is not an expensive option and would be exceedingly quick to deploy.

Language to use

What language do we want to use for this? I'm good with anything in this list:

• Go
• Python
• Typescript

We really don't have too much code to write so I'd prefer python at this point.

[ravenac95]

@conorgil @payaaam Would love your thoughts on the above plan.

[conorgil]

Once donor completes the transaction hopefully squarespace or what not will email the user directly on our behalf. (needs to be derisked)

Stripe can send receipts automatically, so that should be handled: https://stripe.com/docs/receipts

[conorgil]

Keeping infra simple sounds great to me. I have not used GitHub actions before, but excited to learn more about it.

Downside of polling is making sure that we correctly mark each Stripe event as handled/not handled. For example, will Stripe mark the event as delivered as soon as we retrieve it? If yes, then we need to consider how to handle the case where the ClickSend API goes down temporarily, or we need to retry requests, etc to make sure that we process each event. As I type this out, I suppose the same concerns exist even if we received a webhook from Stripe, so maybe no different.

I know that above is just pseudo-code and many of my comments will be addressed, but I'm just writing them down for posterity.

# calculate how many postcards can be sent for a given charge

We need to make sure that we deduct the processing fees and only send postcards for the remainder. Otherwise, I will personally be on the hook for a lot of money :)

sent_addresses_db.insert(sha256(voter.name))

Voters will have the same name, so hash(name) won't work. I imagine that each DB row can have a unique ID and then we can just mark each row as mailed/not mailed.

# Send email to donor send_thank_you_email(

I think we can actually avoid this part. Stripe can send a receipt automatically and that might be "good enough" to allow us to avoid using our own SMTP provider or something like that. We can have our website display the same "We sent postcards to X people! Thanks!" message directly after the Stripe payment returns successfully. Everything else can be done async. Thoughts?

[conorgil]

One other thing we could do is take any of the left over donations and send additional postcards. The postcards will cost us $0.65 for standard mail of $0.98 for priority mail.

Yes! I had the same idea. Let's do it.

[conorgil]

If we choose to use github actions for this we'll need to ensure that no personal information is logged.

That is a VERY good point. We definitely should ensure that no personal information is logged.

What about running a simple Lambda on a cron schedule?

[ravenac95]

Keeping infra simple sounds great to me. I have not used GitHub actions before, but excited to learn more about it.

Downside of polling is making sure that we correctly mark each Stripe event as handled/not handled. For example, will Stripe mark the event as delivered as soon as we retrieve it? If yes, then we need to consider how to handle the case where the ClickSend API goes down temporarily, or we need to retry requests, etc to make sure that we process each event. As I type this out, I suppose the same concerns exist even if we received a webhook from Stripe, so maybe no different.

Ya. I think the same issue applies when using a webhook. In fact, if a webhook fails we don't have retry logic. I think a cron is likely better and simpler to deploy. This also means that it should raise the bar for someone that wants to overload the servers. They will have to spend a lot of money cause you would need to spam through Stripe.

I know that above is just pseudo-code and many of my comments will be addressed, but I'm just writing them down for posterity.

# calculate how many postcards can be sent for a given charge

We need to make sure that we deduct the processing fees and only send postcards for the remainder. Otherwise, I will personally be on the hook for a lot of money :)

Understood.

sent_addresses_db.insert(sha256(voter.name))

Voters will have the same name, so hash(name) won't work. I imagine that each DB row can have a unique ID and then we can just mark each row as mailed/not mailed.

I don't know what the fields exist. This was meant to describe what I felt was a good way to store information anonymously in the db.

# Send email to donor send_thank_you_email(

I think we can actually avoid this part. Stripe can send a receipt automatically and that might be "good enough" to allow us to avoid using our own SMTP provider or something like that. We can have our website display the same "We sent postcards to X people! Thanks!" message directly after the Stripe payment returns successfully. Everything else can be done async. Thoughts?

Sounds good.

[ravenac95]

If we choose to use github actions for this we'll need to ensure that no personal information is logged.

That is a VERY good point. We definitely should ensure that no personal information is logged.

What about running a simple Lambda on a cron schedule?

I just wanted to avoid setting up the deployment system for it haha. Github actions are just immediately deployed but yes we can do that. Can we prevent lambdas from running multiple clones of a lambda in parallel?

[conorgil]

Not being familiar with GitHub actions, I thought another reason to use Lambda or something was to ensure that the ClickSend API token remains secret so that people don't jack up the bill as an attack. However, looks like GH actions handles this use-case with encrypted env vars: docs.

Given that is handled, I suppose the only remaining risk using GH actions is the issue of publicly logging personal information of voters. I am reading up on GH action logging now to understand the risk better.

I agree with you that using GH actions seems the most simplistic deployment route. Would be nice to use it if we can manage the risk.

Can we prevent lambdas from running multiple clones of a lambda in parallel?

Looks like yes. Docs here.

AWS Lambda supports standard rate and cron expressions for frequencies of up to once per minute. CloudWatch Events rate expressions have the following format.

rate(Value Unit)
Where Value is a positive integer and Unit can be minute(s), hour(s), or day(s). For a singular value the unit must be singular (for example, rate(1 day)), otherwise plural (for example, rate(5 days)).

Rate expression examples

Every 5 minutes | rate(5 minutes)
Every hour | rate(1 hour)
Every seven days | rate(7 days)