From 182306a64cd3ea1da5abfd4226b11777e5060d6f Mon Sep 17 00:00:00 2001 From: Aaron Kuehler Date: Wed, 9 Jul 2025 10:23:21 -0400 Subject: [PATCH] Prevent render failure from leaking secrets MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Summary --- > We handle Kubernetes secrets, so it is critical that changes do not cause the contents of any secrets in the template set to be logged.[^1] When a `krane render` task fails it can potentially leak base64 encoded secrets in the clear. Context ---- This was discovered when a developer used Ruby's `Base64.encode64` method to produce a secret value. The value was long enough to produced an encoded string longer than 60 characters. > The returned string ends with a newline character, and if sufficiently long will have one or more embedded newline characters...[^2] > An encoded string returned by Base64.encode64 or Base64.urlsafe_encode64 has an embedded newline character after each 60-character sequence, and, if non-empty, at the end: [^3] This caused krane to dump the base64 encoded secrets to the deployment logs. Deployment tasks take action to prevent secrets leaking in this way, and we can reuse that logic for rendering failures. 🎩 Example before: ``` [INFO][2025-07-09 10:14:03 -0400] [INFO][2025-07-09 10:14:03 -0400] ---------------------------------Phase 1: Initializing render task---------------------------------- [INFO][2025-07-09 10:14:03 -0400] Validating configuration [INFO][2025-07-09 10:14:03 -0400] [INFO][2025-07-09 10:14:03 -0400] -----------------------------------Phase 2: Rendering template(s)----------------------------------- [ERROR][2025-07-09 10:14:03 -0400] Failed to render secret.yaml.erb [INFO][2025-07-09 10:14:03 -0400] [INFO][2025-07-09 10:14:03 -0400] ------------------------------------------Result: FAILURE------------------------------------------- [FATAL][2025-07-09 10:14:03 -0400] Invalid template: secret.yaml.erb [FATAL][2025-07-09 10:14:03 -0400] > Error message: [FATAL][2025-07-09 10:14:03 -0400] ( secret.yaml.erb): could not find expected ':' while scanning a simple key at line 10 column 1 [FATAL][2025-07-09 10:14:03 -0400] > Template content: [FATAL][2025-07-09 10:14:03 -0400] [FATAL][2025-07-09 10:14:03 -0400] apiVersion: v1 [FATAL][2025-07-09 10:14:03 -0400] kind: Secret [FATAL][2025-07-09 10:14:03 -0400] metadata: [FATAL][2025-07-09 10:14:03 -0400] name: invalid-secret [FATAL][2025-07-09 10:14:03 -0400] type: Opaque [FATAL][2025-07-09 10:14:03 -0400] data: [FATAL][2025-07-09 10:14:03 -0400] username: YWRtaW4= [FATAL][2025-07-09 10:14:03 -0400] password: KioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioqKioq [FATAL][2025-07-09 10:14:03 -0400] KioqKioqKioqKioqKioqKioqKioqKioqKg== [FATAL][2025-07-09 10:14:03 -0400] [FATAL][2025-07-09 10:14:03 -0400] ``` Example After: ``` [INFO][2025-07-09 10:27:04 -0400] [INFO][2025-07-09 10:27:04 -0400] ---------------------------------Phase 1: Initializing render task---------------------------------- [INFO][2025-07-09 10:27:04 -0400] Validating configuration [INFO][2025-07-09 10:27:04 -0400] [INFO][2025-07-09 10:27:04 -0400] -----------------------------------Phase 2: Rendering template(s)----------------------------------- [INFO][2025-07-09 10:27:04 -0400] [INFO][2025-07-09 10:27:04 -0400] ------------------------------------------Result: FAILURE------------------------------------------- [FATAL][2025-07-09 10:27:04 -0400] Invalid template: secret.yaml.erb [FATAL][2025-07-09 10:27:04 -0400] > Error message: [FATAL][2025-07-09 10:27:04 -0400] ( secret.yaml.erb): could not find expected ':' while scanning a simple key at line 10 column 1 [FATAL][2025-07-09 10:27:04 -0400] > Template content: Suppressed because it may contain a Secret ``` [^1]: https://github.com/Shopify/krane/blob/72ee07a311442751535b87a23ea843b71190e618/CONTRIBUTING.md?plain=1#L44 [^2]: https://ruby-doc.org/3.4.1/gems/base64/Base64.html#method-i-encode64 [^3]: https://ruby-doc.org/3.4.1/gems/base64/Base64.html#module-Base64-label-Newlines --- lib/krane/render_task.rb | 20 ++++++++------------ test/fixtures/invalid/secret.yaml.erb | 10 ++++++++++ test/integration/render_task_test.rb | 13 +++++++++++++ 3 files changed, 31 insertions(+), 12 deletions(-) create mode 100644 test/fixtures/invalid/secret.yaml.erb diff --git a/lib/krane/render_task.rb b/lib/krane/render_task.rb index 0707f373f..14f9224f9 100644 --- a/lib/krane/render_task.rb +++ b/lib/krane/render_task.rb @@ -8,6 +8,8 @@ module Krane # Render templates class RenderTask + include Krane::TemplateReporting + # Initializes the render task # # @param logger [Object] Logger object (defaults to an instance of Krane::FormattedLogger) @@ -67,7 +69,12 @@ def render_templates(stream, template_sets) count rescue Krane::InvalidTemplateError => exception - log_invalid_template(exception) + record_invalid_template( + logger: @logger, + err: exception.to_s, + filename: exception.filename, + content: exception.content + ) raise end @@ -105,16 +112,5 @@ def validate_configuration(template_sets) raise Krane::TaskConfigurationError, "Configuration invalid: #{errors.join(', ')}" end end - - def log_invalid_template(exception) - @logger.error("Failed to render #{exception.filename}") - - debug_msg = ColorizedString.new("Invalid template: #{exception.filename}\n").red - debug_msg += "> Error message:\n#{FormattedLogger.indent_four(exception.to_s)}" - if exception.content - debug_msg += "\n> Template content:\n#{FormattedLogger.indent_four(exception.content)}" - end - @logger.summary.add_paragraph(debug_msg) - end end end diff --git a/test/fixtures/invalid/secret.yaml.erb b/test/fixtures/invalid/secret.yaml.erb new file mode 100644 index 000000000..911b171f2 --- /dev/null +++ b/test/fixtures/invalid/secret.yaml.erb @@ -0,0 +1,10 @@ +<%- invalid_secret_with_newlines = Base64.encode64("*" * 70) -%> + +apiVersion: v1 +kind: Secret +metadata: + name: invalid-secret +type: Opaque +data: + username: YWRtaW4= + password: <%= invalid_secret_with_newlines %> diff --git a/test/integration/render_task_test.rb b/test/integration/render_task_test.rb index 2d94c386a..e3715ee3b 100644 --- a/test/integration/render_task_test.rb +++ b/test/integration/render_task_test.rb @@ -234,6 +234,19 @@ def test_render_runtime_error_when_rendering ], in_order: true) end + def test_render_runtime_error_when_rendering_secrets + render = build_render_task( + File.join(fixture_path('invalid'), 'secret.yaml.erb') + ) + assert_render_failure(render.run(stream: mock_output_stream)) + assert_logs_match_all([ + /Invalid template: secret.yaml.erb/, + "> Error message:", + /\( secret.yaml.erb\): could not find expected ':' while scanning a simple key at line \d+ column \d+/, + "> Template content: Suppressed because it may contain a Secret", + ], in_order: true) + end + def test_render_empty_template_dir tmp_dir = Dir.mktmpdir render = build_render_task(tmp_dir)