Tutorial part 2 by khamilowicz · Pull Request #4 · membraneframework-labs/ex_broadcaster

khamilowicz · 2026-07-15T13:21:49Z

Second part of the tutorial, describes deployment, monitoring, and CDN setup.

Do not merge yet, requires changes from membraneframework/membrane_transcoder_plugin#25

… device plugin and network settings, add K8s manifests for secrets, cms, service and deployments

Prevents accidentally committing terraform.tfstate, .pem keys, and tfvars files that may contain credentials or infrastructure state. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

varsill · 2026-07-20T15:02:27Z

      {:plug, "~> 1.14"},
      {:ex_aws_s3, "~> 2.5"},
      {:hackney, ">= 0.0.0"},
-      {:credo, ">= 0.0.0", only: :dev, runtime: false}


why this only: :dev part was removed? :D

varsill · 2026-07-20T15:08:39Z

-      bitrate: 4_000_000,
-      framerate: {30, 1}
+      framerate: {30, 1},
+      bitrate: %VariableBitrate{average_bitrate: 5_000_000, max_bitrate: 6_000_000}


Let's make max_bitrate a value derived from average_bitrate.
For lifestreams it's good to keep almost CBR so I would say that we should set max_bitrate = 1.1*max_bitrate at most. We can even consider using a real CBR here.

varsill · 2026-07-21T08:26:12Z

How about we added some bucket lifecycle configuration to delete objects after given period of time?
(I am afraid people who read the tutorial might forget to remove the objects on their own and will be charged with storage costs)

Also - wouldn't it be better if we allowed only Cloudfront to access S3?
Now this bucket has their access policy set in "chmod 777" manner :D
If we want to preserve it like that, let's add a comment that it's not advised to use this code on production.

varsill · 2026-07-21T08:27:25Z

@@ -0,0 +1,79 @@
+variable "aws_region" {
+  description = "AWS region (must match the ambient provider config)"


I think provider already uses this variable, so we can remove this "(must match the ambient provider config)" part.

varsill · 2026-07-21T08:28:15Z

@@ -0,0 +1,19 @@
+terraform {


consider adding required_version key as well

varsill · 2026-07-21T08:29:07Z

+    rules = [
+      {
+        rulePriority = 1
+        description  = "Expire untagged images after 7 days"


Don't we want to add some cleanup policy for tagged images as well?
WDYT?

varsill · 2026-07-21T08:37:42Z

+  }
+
+  # No inbound SSH rule: SSH access goes over an SSM Session Manager tunnel
+  # (see ssh.tf), not a direct port 22 opening.


I don't think ssh.tf exists 🤔

varsill · 2026-07-21T08:39:35Z

+  count               = var.gpu_enabled ? 1 : 0
+  alarm_name          = "ex-broadcaster-gpu-utilization-high"
+  namespace           = "ExBroadcaster"
+  metric_name         = "utilization_gpu"


Shouldn't we use nvidia_smi_utilization_gpu?
https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch-Agent-NVIDIA-GPU.html

varsill · 2026-07-21T09:01:52Z

+
+  launch_template {
+    id      = aws_launch_template.ex_broadcaster.id
+    version = "$Latest"


I think it would be better to do:

Suggested change

version = "$Latest"

version = "aws_launch_template.ex_broadcaster.latest_version"

since with $Latest, even when the template changes, this blueprint stays the same (while it should also be updated).

varsill · 2026-07-27T08:57:00Z

-which Membrane element uses under the hood.
-In the next chapters we will focus on deploying the application in the cloud environment
-so you won’t need to run the application locally, so this requirement won’t apply anymore.
+For more information, you can take a look at [membrane_transcoder_plugin](https://hexdocs.pm/membrane_transcoder_plugin) which provides hardware-accelerated transcoding capabilities via the `membrane_vk_video_plugin` dependency.


I would leave a direct link to the https://crates.io/crates/vk-video somewhere here

varsill · 2026-07-27T09:02:15Z

      {:membrane_core, "~> 1.2"},
-      {:membrane_vk_video_plugin, "~> 0.2.1"},
+      {:membrane_transcoder_plugin, "~> 0.4.0"},
+      {:membrane_vk_video_plugin, "> 0.0.0"},


[NIT] Wouldn't it be better to do:

Suggested change

{:membrane_vk_video_plugin, "> 0.0.0"},

{:membrane_vk_video_plugin, "~> 0.2.0"},

?
Also, I don't think we should it should always be added i.e. if somebody is not on Linux with Vulkan support, than he shouldn't add it and we should mention that

varsill · 2026-07-27T09:03:15Z

 In this chapter we will create an Elixir application and run it locally.
-Since the application will be performing hardware-accelerated video transcoding with the use of Vulkan Video Extensions,
-you need a Linux machine with a Vulkan-capable GPU (NVIDIA or AMD) with Mesa drivers and Vulkan Video extension support.
+The application can benefit from hardware-accelerated video transcoding via Vulkan Video native acceleration on Linux machines with a Vulkan-capable GPU (NVIDIA or AMD) with Mesa drivers and Vulkan Video extension support. It is not required though, `Membrane.Transcoder` will adapt to your environment.


I am not sure about this "Membrane.Transcoder will adapt to your environment" though - if I remember correctly, it's user's responsibility to either add or not add membrane_transcoder_plugin's optional dependency to membrane_vk_video_plugin

varsill · 2026-07-27T09:04:59Z

+      width: 1920,
+      height: 1080,
+      framerate: {30, 1},
+      bitrate: %VariableBitrate{average_bitrate: 5_000_000, max_bitrate: 6_000_000}


Don't we want to use ConstantBitrate since it's meant to be used for a livestream?

varsill · 2026-07-27T09:07:20Z

@@ -339,7 +385,7 @@ graph LR

    subgraph "Video branch"
        H264in["Membrane.H264.Parser\n(annexb · au-aligned)"]


I don't think we need this parser, Membrane.Transcoder will inject it automatically if it's necessary

varsill · 2026-07-27T09:14:32Z

+**Key improvement in v0.4.0**: Instead of creating three separate `Membrane.Transcoder` instances (one per resolution),
+we now use a single transcoder with three output pads, each configured with different output stream formats.
+This is more efficient as it allows the transcoder to share resources between the different output variants.


Let's remove this one as well ;)

varsill · 2026-07-27T09:15:27Z

+
 We only need to `terminate: normal` when the end-of-stream signal arrives at `:hls_sink` sink element.

+### Using membrane_vk_video_plugin directly


Do we really want to have this section here?
This first chapter is already quite long so I don't think we need to make it even longer

varsill · 2026-07-27T09:15:56Z

+    })
+```
+
+#### When to use membrane_vk_video_plugin directly


Same as above

varsill · 2026-07-27T09:16:53Z

 Alternatively, paste it into the [hls.js demo player](https://hlsjs.video-dev.org/demo/) (local HTTP server is configure to allow all CORS origins).
 You should be able to see that the resolution changes throughout based on your network condition or even manually change it.

+### Streaming from OBS Studio instead of FFmpeg


This section also looks a little bit out of the scope for this tutorial 🤔

varsill · 2026-07-27T09:17:09Z

+- The bottom-right status bar in OBS shows the live connection state; "Reconnecting..." there usually means the
+  server isn't reachable at the address/port you configured, not a codec problem.
+
+### Watching the stream in VLC


Same as above

varsill · 2026-07-27T09:24:00Z

+Approval is often instant but can take up to a day or two for larger increases — request it first, then move on to
+the rest of the setup while you wait.
+
+You don't have to sit around waiting for it, though: set `gpu_enabled = false` (in `terraform.tfvars` or via


Isn't it variables.tf instead of terraform.tfvars?

varsill · 2026-07-27T09:25:00Z

+`terraform apply -var="gpu_enabled=false"`) and the ASG falls back to a `c6i.large` (no GPU, no Vulkan Video quota needed) instead
+of `g6.xlarge`, so you can bring up the rest of the stack and exercise it end-to-end immediately. Vulkan Video
+hardware acceleration just won't be available on that fleet, so `Membrane.Transcoder` falls back to software
+encoding — fine for getting everything wired up, but switch back to `gpu_enabled = true` (the default) once your


This part of sentence sounds too LLM-like to me, let's rephrase it :D

varsill · 2026-07-27T09:29:00Z

+
+### The S3 bucket
+
+`s3.tf` provisions the same bucket chapter 1's `S3Storage` writes to, now as part of the infrastructure instead of


Will the Elixir application be "automatically" aware of this bucket name?

varsill · 2026-07-27T09:32:17Z

+}
+```
+
+Two things worth flagging, both left as known limitations rather than hidden:


Suggested change

Two things worth flagging, both left as known limitations rather than hidden:

Two things worth flagging:

varsill · 2026-07-27T09:34:22Z

+A couple of details worth calling out: the container's stdout/stderr go straight to CloudWatch Logs via Docker's
+own `awslogs` log driver, no sidecar needed. And port 8080 — the development HTTP server from chapter 1 — is only
+bound to `127.0.0.1`, not exposed to the NLB or the internet; in production, HLS is served from S3 (directly, or
+through a CDN in front of it), so the in-process HTTP server has no reason to be reachable from outside the instance.


Let's enumerate it and make it sound more human friendly ;)

varsill · 2026-07-27T09:35:31Z

+OBS's connection indicator (bottom-right) turning green, with a steady bitrate and no dropped-frames warning,
+means the NLB accepted the connection and RTMP ingest is flowing into the ASG.
+
+### Watching the stream with VLC


Do we really need to use VLC? In most cases viewers view HLS stream directly in browser, I would say that hls.js is good enough for the sake of testing the output

varsill · 2026-08-12T09:08:32Z

      {:ex_aws_s3, "~> 2.5"},
      {:hackney, ">= 0.0.0"},
-      {:credo, ">= 0.0.0", only: :dev, runtime: false}
+      {:credo, ">= 0.0.0", only: :dev, runtime: false, override: true}


Why would we need this override: true? 🤔

varsill · 2026-08-12T09:11:12Z

+  Each output pad also carries its own target bitrate (`Membrane.Transcoder.Video.VariableBitrate`),
+  so the transcoder encodes a proper bitrate ladder alongside the resolution ladder instead of
+  leaving every variant at the encoder's default rate control.


I think it's quite obvious that it happens this way, so let's remove this comment

varsill · 2026-08-12T13:31:29Z

+docker buildx build \
+  --platform linux/amd64 \
+  -t <account-id>.dkr.ecr.<your-aws-region>.amazonaws.com/ex-broadcaster:latest \
+  --push .


Don't we need to --build-context transcoder_plugin since the Dockerfile has COPY --from=transcoder_plugin?

varsill · 2026-08-12T13:32:55Z

+GPU_DOCKER_ARGS=()
+
+apt-get update -y
+apt-get install -y linux-headers-aws ubuntu-drivers-common gcc-12 nvidia-driver-595 nvidia-utils-595


Do we need to install nvidia-driver no matter if ${gpu_enabled}" = "true"?

varsill · 2026-08-12T13:35:13Z

+ARG ELIXIR_VERSION=1.17.3
+ARG OTP_VERSION=27.1.2


Are these two used at all? 🤔

varsill · 2026-08-12T13:39:16Z

+  | 1080p | 1920 × 1080 | 5.0 Mbps / 6.0 Mbps  |
+  | 720p  | 1280 × 720  | 2.8 Mbps / 3.5 Mbps  |
+  | 480p  | 854 × 480   | 1.4 Mbps / 1.75 Mbps |


I don't think these bitrates are corresponding to the ones in code (we have @max_bitrate_factor equal to 1.1 and 5*1.1 is 5.5, not 6 :D)

varsill · 2026-08-12T13:42:32Z


  alias ExBroadcaster.Storages.S3Storage
  alias Membrane.HTTPAdaptiveStream.Storages.FileStorage
+  alias Membrane.RTMPServer, as: RTMPServer


I don't think this as: RTMPServer is needed :D

varsill and others added 9 commits May 8, 2026 12:59

WIP - add Terraform config for S3, EKS, custom EC2 runner with NVidia…

d9ff986

… device plugin and network settings, add K8s manifests for secrets, cms, service and deployments

Use membrane_transcoder instead of membrane_vk_video

07b6d4d

Merge branch 'chapter2' into vk-transcoder-update

5299737

Remove k8s

6de45b7

Add cloudwatch, some tweaks

293a8dc

fix

5a989b8

Add tutorial

ce9fd38

Optionally set gpu enabled

b10c389

Add CDN

46b8ec1

khamilowicz changed the title ~~Replace EKS/Kubernetes deployment with a plain EC2 Auto Scaling Group + CDN~~ Tutorial part 2 Jul 15, 2026

khamilowicz requested a review from varsill July 15, 2026 13:26

Ignore Terraform state and secret-bearing files

658dba6

Prevents accidentally committing terraform.tfstate, .pem keys, and tfvars files that may contain credentials or infrastructure state. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

khamilowicz force-pushed the remove-kubernetes branch 4 times, most recently from d2ccc57 to 9594d7c Compare July 15, 2026 14:03

Add bitrate

8ce9008

khamilowicz force-pushed the remove-kubernetes branch from 9594d7c to 8ce9008 Compare July 16, 2026 09:16

varsill requested changes Jul 21, 2026

View reviewed changes

varsill requested changes Jul 27, 2026

View reviewed changes

khamilowicz force-pushed the remove-kubernetes branch 2 times, most recently from ce68163 to 593bd02 Compare August 3, 2026 13:13

Fix GPU setup

6775378

khamilowicz force-pushed the remove-kubernetes branch from 593bd02 to 6775378 Compare August 3, 2026 13:18

khamilowicz requested a review from varsill August 3, 2026 13:18

varsill requested changes Aug 12, 2026

View reviewed changes

varsill reviewed Aug 12, 2026

View reviewed changes

		@@ -0,0 +1,79 @@
		variable "aws_region" {
		description = "AWS region (must match the ambient provider config)"

	version = "$Latest"
	version = "aws_launch_template.ex_broadcaster.latest_version"

	{:membrane_vk_video_plugin, "> 0.0.0"},
	{:membrane_vk_video_plugin, "~> 0.2.0"},

		@@ -339,7 +385,7 @@ graph LR

		subgraph "Video branch"
		H264in["Membrane.H264.Parser\n(annexb · au-aligned)"]


		We only need to `terminate: normal` when the end-of-stream signal arrives at `:hls_sink` sink element.

		### Using membrane_vk_video_plugin directly


		### The S3 bucket

		`s3.tf` provisions the same bucket chapter 1's `S3Storage` writes to, now as part of the infrastructure instead of

	Two things worth flagging, both left as known limitations rather than hidden:
	Two things worth flagging:

		ARG ELIXIR_VERSION=1.17.3
		ARG OTP_VERSION=27.1.2

Uh oh!

Conversation

khamilowicz commented Jul 15, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

khamilowicz commented Jul 15, 2026 •

edited

Loading