Skip to content

Fix PGCR route decode for plain JSON storage (API-1N) #149

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
owens1127 merged 1 commit into from
Jun 23, 2026
Merged

Fix PGCR route decode for plain JSON storage (API-1N) #149

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 2 additions & 6 deletions src/routes/pgcr.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,12 +7,9 @@ import {
import { ErrorCode } from "@/schema/errors/ErrorCode"
import { zBigIntString } from "@/schema/input"
import { zInt64 } from "@/schema/output"
import { getRawCompressedPGCR } from "@/services/pgcr"
import { gunzipSync } from "bun"
import { decodePgcrPayload, getRawCompressedPGCR } from "@/services/pgcr"
import { z } from "zod"

const decoder = new TextDecoder()

export const pgcrRoute = new RaidHubRoute({
method: "get",
description: `Get a raw post game carnage report by instanceId.
Expand Down Expand Up @@ -46,8 +43,7 @@ Useful if you need to access PGCRs when Bungie's API is down.`,
return RaidHubRoute.fail(ErrorCode.PGCRNotFoundError, { instanceId })
}

const decompressed = gunzipSync(result.data)
const pgcr = JSON.parse(decoder.decode(decompressed)) as RaidHubPostGameCarnageReport
const pgcr = JSON.parse(decodePgcrPayload(result.data)) as RaidHubPostGameCarnageReport
pgcr.activityDetails.instanceId = BigInt(pgcr.activityDetails.instanceId)
pgcr.entries.forEach(entry => {
entry.characterId = BigInt(entry.characterId)
Expand Down
51 changes: 13 additions & 38 deletions src/services/pgcr.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,43 +1,18 @@
import { getFixturePool } from "@/lib/test-fixture-db"
import { gzipPgcrJson } from "@/lib/test-minimal-pgcr"
import { afterAll, beforeAll, describe, expect, test } from "bun:test"
import { z } from "zod"
import { getRawCompressedPGCR } from "./pgcr"
import { buildMinimalRaidHubPgcrJson, gzipPgcrJson } from "@/lib/test-minimal-pgcr"
import { describe, expect, test } from "bun:test"
import { decodePgcrPayload } from "./pgcr"

const fixtureDb = getFixturePool()
const fixturePgcrInstanceId = "999000000703"
describe("decodePgcrPayload", () => {
const instanceId = "999000000704"

beforeAll(async () => {
await fixtureDb.query(`DELETE FROM raw.pgcr WHERE instance_id = $1::bigint`, [
fixturePgcrInstanceId
])
await fixtureDb.query(
`INSERT INTO raw.pgcr (instance_id, data, date_crawled) VALUES ($1::bigint, $2, NOW())`,
[fixturePgcrInstanceId, gzipPgcrJson(fixturePgcrInstanceId)]
)
})

afterAll(async () => {
await fixtureDb.query(`DELETE FROM raw.pgcr WHERE instance_id = $1::bigint`, [
fixturePgcrInstanceId
])
})

describe("getRawCompressedPGCR", () => {
test("returns the correct shape", async () => {
const data = await getRawCompressedPGCR(fixturePgcrInstanceId).catch(console.error)
test("decodes gzip-compressed JSON", () => {
const json = decodePgcrPayload(gzipPgcrJson(instanceId))
expect(JSON.parse(json).activityDetails.instanceId).toBe(instanceId)
})

const parsed = z
.object({
data: z.instanceof(Buffer)
})
.strict()
.safeParse(data)
if (!parsed.success) {
console.error(parsed.error.errors)
expect(parsed.error.errors).toEqual([])
} else {
expect(parsed.success).toBe(true)
}
test("decodes plain JSON", () => {
const plain = Buffer.from(JSON.stringify(buildMinimalRaidHubPgcrJson(instanceId)))
const json = decodePgcrPayload(plain)
expect(JSON.parse(json).activityDetails.instanceId).toBe(instanceId)
})
})
16 changes: 16 additions & 0 deletions src/services/pgcr.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,4 +1,20 @@
import { pgReader } from "@/integrations/postgres"
import { gunzipSync } from "bun"

const decoder = new TextDecoder()

/** Gzip magic bytes — matches Services `log-raw-pgcr` and Hermes storage (plain JSON when absent). */
function isGzipCompressed(data: Buffer): boolean {
return data.length >= 2 && data[0] === 0x1f && data[1] === 0x8b
}

/** Decode PGCR payload stored as gzip (legacy raw.pgcr) or plain JSON (pgcr table). */
export function decodePgcrPayload(data: Buffer): string {
if (isGzipCompressed(data)) {
return decoder.decode(gunzipSync(data))
}
return decoder.decode(data)
Comment on lines +12 to +16

@sentry sentry Bot Jun 23, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: The code expects a Buffer from the BYTEA database column but receives a string. Passing this string to TextDecoder.decode() will cause a TypeError at runtime.
Severity: CRITICAL

Suggested Fix

Configure a type parser for the BYTEA type (OID 17) in src/integrations/postgres/parsers.ts. This will ensure that BYTEA columns are correctly parsed into Buffer objects, matching the expected data: Buffer type annotation and preventing the runtime error.

Prompt for AI Agent 
Review the code at the location below. A potential bug has been identified by an AI
agent. Verify if this is a real issue. If it is, propose a fix; if not, explain why it's
not valid.

Location: src/services/pgcr.ts#L12-L16

Potential issue: The `getRawCompressedPGCR` function expects the `BYTEA` data column
from the database to be a `Buffer`. However, the `node-postgres` driver is not
configured with a type parser for `BYTEA` columns (type 17), so it returns a
backslash-escaped string by default. This string is passed to `decodePgcrPayload`, which
then calls `TextDecoder.decode()`. Since `TextDecoder.decode()` does not accept a
string, it will throw a `TypeError`, causing a runtime crash whenever the application
attempts to retrieve a plain JSON PGCR.

Also affects:

  • src/integrations/postgres/parsers.ts
  • src/services/pgcr.ts:20-23

Did we get this right? 👍 / 👎 to inform future reviews.

}

export async function getRawCompressedPGCR(instanceId: bigint | string) {
return await pgReader.queryRow<{
Expand Down
Loading