Skip to content

fix(release): update the deployment script #138

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Ayush8923 wants to merge 2 commits into
base: main
Choose a base branch
from
+42 −26
Open

fix(release): update the deployment script #138

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
3cf441f
fix(release): update the release script
Ayush8923 Jul 21, 2026
ad915f9
fix(ci): fix the formatting
Ayush8923 Jul 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/cd-production.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ jobs:
INSTANCE_ID: ${{ secrets.EC2_INSTANCE_ID }}
BUILD_DIRECTORY: ${{ secrets.BUILD_DIRECTORY }}
run: |
DEPLOY_CMD="cd ${BUILD_DIRECTORY} && git fetch --tags origin && git checkout --force ${{ github.sha }} && docker compose build && docker compose run --rm --entrypoint '' backend uv run --frozen alembic upgrade head && docker compose up -d --remove-orphans && docker image prune -f"
DEPLOY_CMD="cd ${BUILD_DIRECTORY} && git fetch --tags origin && git checkout --force ${{ github.sha }} && docker compose build && docker compose run --rm --entrypoint '' backend uv run --frozen alembic upgrade head && docker compose up -d --remove-orphans && docker image prune -f && docker builder prune -af"

@coderabbitai coderabbitai Bot Jul 21, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🩺 Stability & Availability | 🟠 Major | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
set -euo pipefail

for f in .github/workflows/cd-production.yml .github/workflows/cd-staging.yml; do
  echo "===== $f ====="
  nl -ba "$f" | sed -n '1,220p'
  echo
done

echo "===== search for SSM polling / timeout logic ====="
rg -n "wait|poll|timeout|ssm|builder prune|docker builder prune|docker image prune|DEPLOY_CMD" .github/workflows . -g '!**/node_modules/**'

Repository: ProjectTech4DevAI/kaapi-guardrails

Length of output: 260

🏁 Script executed:

#!/bin/bash
set -euo pipefail

for f in .github/workflows/cd-production.yml .github/workflows/cd-staging.yml; do
  echo "===== $f ====="
  cat -n "$f" | sed -n '1,220p'
  echo
done

echo "===== search for polling / timeout logic ====="
rg -n "wait|poll|timeout|ssm|builder prune|docker builder prune|docker image prune|DEPLOY_CMD|send-command|get-command-invocation" .github/workflows . -g '!**/node_modules/**'

Repository: ProjectTech4DevAI/kaapi-guardrails

Length of output: 15127

🌐 Web query:

AWS SSM send-command get-command-invocation status poll timeout default command continues after client stops polling

💡 Result:

When you use AWS Systems Manager (SSM) Run Command, the process is asynchronous [1]. Stopping your client-side polling of get-command-invocation has no effect on the execution of the command on the managed node [1]. Key points regarding this behavior: 1. Asynchronous Execution: The send-command API initiates the command and returns a command ID, but it does not wait for the command to finish [1]. The command continues to run independently on the target instance regardless of what happens to the client that issued it [1][2]. 2. Polling vs. Execution: get-command-invocation is simply a mechanism to query the current status and output of a command that is already running or has already completed on the managed node [3][4]. Your client-side polling loop (or the CLI/SDK waiters) is only responsible for observing the state; it does not control the command's lifecycle [1][5][6]. 3. Command Timeouts: The lifecycle of the command is governed by parameters defined during the send-command call, specifically TimeoutSeconds (for delivery) and, if defined in the SSM document, Execution Timeout [2][7]. If these limits are reached, the SSM Agent on the managed node will terminate the execution and report the status as DeliveryTimedOut or ExecutionTimedOut, even if your client has stopped polling entirely [2][3]. In summary, the command on the target node is completely decoupled from your polling client. It will continue to execute until it finishes, reaches a terminal state (such as Success, Failed, or TimedOut), or is explicitly cancelled by another call to the cancel-command API [2][3].

Citations:

Extend the SSM wait window or cap the prune step.

docker builder prune -af runs inside the SSM command, but both workflows only poll for about 5 minutes (20 × 15s). If cleanup runs longer, the GitHub job fails while the remote command may still finish successfully, which creates false deployment failures and encourages unsafe retries.

  • .github/workflows/cd-production.yml#L34
  • .github/workflows/cd-staging.yml#L33
📍 Affects 2 files
  • .github/workflows/cd-production.yml#L34-L34 (this comment)
  • .github/workflows/cd-staging.yml#L33-L33
🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.github/workflows/cd-production.yml at line 34, Update the SSM deployment
commands in .github/workflows/cd-production.yml line 34 and
.github/workflows/cd-staging.yml line 33 to prevent docker builder prune -af
from exceeding the workflow’s polling window: either extend the associated SSM
wait/poll duration beyond five minutes or cap the prune step while preserving
successful deployment completion.

CMD_ID=$(aws ssm send-command \
--instance-ids "$INSTANCE_ID" \
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/cd-staging.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ jobs:
INSTANCE_ID: ${{ secrets.STAGING_EC2_INSTANCE_ID }}
BUILD_DIRECTORY: ${{ secrets.BUILD_DIRECTORY }}
run: |
DEPLOY_CMD='sudo -iu ec2-user bash -lc \"cd '"${BUILD_DIRECTORY}"' && git fetch origin && git reset --hard origin/main && docker compose build && docker compose run --rm --entrypoint \\\"\\\" backend uv run --frozen alembic upgrade head && docker compose up -d --remove-orphans && docker image prune -f\"'
DEPLOY_CMD='sudo -iu ec2-user bash -lc \"cd '"${BUILD_DIRECTORY}"' && git fetch origin && git reset --hard origin/main && docker compose build && docker compose run --rm --entrypoint \\\"\\\" backend uv run --frozen alembic upgrade head && docker compose up -d --remove-orphans && docker image prune -f && docker builder prune -af\"'
CMD_ID=$(aws ssm send-command \
--instance-ids "$INSTANCE_ID" \
Expand Down
64 changes: 40 additions & 24 deletions backend/notebooks/analysis-pii.py
View file
Open in desktop

@Ayush8923 Ayush8923 Jul 21, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this PR only includes formatting-related CI fixes.

Original file line number Diff line number Diff line change
Expand Up @@ -11,22 +11,21 @@ def _():
import io
from app.core.validators.topic_relevance_llm import TopicRelevanceLLM
from app.core.validators.pii_remover import PIIRemover
from guardrails.validators import (
FailResult,
PassResult
)
from guardrails.validators import FailResult, PassResult
from sklearn.metrics import classification_report, confusion_matrix

return PIIRemover, classification_report, confusion_matrix, io, mo, pd


@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
# Expected Format of the data
- query : The text message that you want to run against the validators
- guardrail_triggered : name of the validators you expect to catch this query. In case of queries that should get caught by multiple validators, you can separate them with a "+" sign. for instance : PII + topic_relevance
""")
"""
)
return


Expand All @@ -50,9 +49,11 @@ def _(file_picker, io, mo, pd):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
# Calculate PII match
""")
"""
)
return


Expand All @@ -69,14 +70,17 @@ def _(PIIRemover):
"IN_VOTER",
"IN_PASSPORT",
"IN_VEHICLE_REGISTRATION",
"URL"],
"URL",
],
)
return (pii_validator,)


@app.cell
def _(pii_validator):
_result = pii_validator._validate("hey buddy budgeting kya hota haiii muje asan trike se smjao with examples")
_result = pii_validator._validate(
"hey buddy budgeting kya hota haiii muje asan trike se smjao with examples"
)
_result.outcome == "fail"
return

Expand All @@ -87,7 +91,7 @@ def actual_label(input):
return "PII" in input

user_df["pii_true"] = user_df["guardrail_triggered"].apply(actual_label)
user_df[["query", "guardrail_triggered" ,"pii_true"]]
user_df[["query", "guardrail_triggered", "pii_true"]]
return


Expand All @@ -98,15 +102,17 @@ def is_pii(message):
return result.outcome == "fail"

user_df["pii_pred"] = user_df["query"].apply(is_pii)
user_df[["query","guardrail_triggered","pii_true","pii_pred"]]
user_df[["query", "guardrail_triggered", "pii_true", "pii_pred"]]
return


@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
# Scores
""")
"""
)
return


Expand All @@ -118,9 +124,11 @@ def _(classification_report, user_df):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
# Confusion Matrix
""")
"""
)
return


Expand All @@ -134,9 +142,11 @@ def _(confusion_matrix, user_df):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
## False Negatives
""")
"""
)
return


Expand All @@ -148,9 +158,11 @@ def _(user_df):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
## True Positives
""")
"""
)
return


Expand All @@ -162,9 +174,11 @@ def _(user_df):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
## False Positives
""")
"""
)
return


Expand All @@ -176,9 +190,11 @@ def _(user_df):

@app.cell(hide_code=True)
def _(mo):
mo.md(r"""
mo.md(
r"""
## True Negatives
""")
"""
)
return


Expand Down