Skip to content

Feat/liveness detection - #1157

Open
kanth071 wants to merge 4 commits into
vicharanashala:mainfrom
kanth071:feat/liveness-detection
Open

kanth071 wants to merge 4 commits into
vicharanashala:mainfrom
kanth071:feat/liveness-detection

Conversation

@kanth071

Copy link
Copy Markdown

Description

This PR implements real-time webcam liveness detection (stillness, freeze-frame, and looking away signals) on top of the explainable violation metadata schema introduced in PR 1.

Project Philosophy Alignment: "Trust, Not Surveillance"

In alignment with the core project philosophy of "trust, not surveillance," this change ensures that:

  • No Single Frame Penalties: Violations are never fired on a single bad frame. The LivenessDetector requires a sustained window (12s for stillness, 5s for looking away) and a minimum of 8 consecutive frames before triggering.
  • Instructors Get Context: Every violation carries a plain-English reason showing exactly why it was flagged.
  • Full Coordinator Control: The new liveness detector is fully toggleable per-course, giving coordinators complete control.

Changes Introduced:

  • LivenessDetector Component: Monitors the primary face for three signals:
    1. Stillness: Tracks nose keypoint drift over a 12s rolling window. If movement is below a threshold, it flags static photo attempts.
    2. Freeze-frame: Calculates coordinate variance over 15s to detect frozen video loops.
    3. Looking Away: Reuses existing keypoint geometry to detect sustained head turns (> 5s).
  • Proctoring Settings Toggle: Registered the livenessDetection toggle on the backend settings repository (ISettingRepository.ts) and the frontend setting modal (EditProctoringModal.tsx).
  • Server-Side Gate Security: Extended the backend controller to throw a ForbiddenError if a student submits a liveness/looking-away anomaly for a course where liveness detection is disabled.
  • Central useViolationReporter: Built a hook to draw frame canvas snapshots and submit them to the backend with cooldown protection (60s limit per violation type).
  • Teacher Explanation Badge: Mapped explanations onto the AnomaliesList table using a color-coded hover tooltip displaying the reason and confidence score.
  • Math Test Coverage: Added unit tests for liveness, stillness, and head-turn math in Anomaly.test.ts.

Related Issues

Closes #

kanth071 added 3 commits July 10, 2026 23:31
…emoryReplSet

- Replace MongoMemoryServer (standalone, no transactions) with
  MongoMemoryReplSet so MongoDB transactions used by CourseService and
  AnomalyService work correctly in the test environment.

- Fix InversifyJS container wiring for tests: use unbind()+bind() instead
  of the removed rebind() chain, and rebind GLOBAL_TYPES.Database so a
  fresh MongoDatabase is constructed with the correct in-memory URI
  (bypassing the hardcoded ssl/tls options).

- Break circular dep: import CourseSetting directly from its transformer
  file instead of the setting barrel index to avoid the
  'CourseSetting is not a constructor' Vite SSR error caused by the
  courses <-> setting circular import chain.

- Fix multipart request format: send NewAnomalyData fields as individual
  form fields (.field('courseId', ...)) rather than a single JSON-encoded
  'data' field that bypassed class-validator.

- Mock CloudStorageService.uploadAnomaly to prevent calls to GCS during
  tests (no credentials in CI).

- Replace bogus error scenario ('Input buffer contains unsupported image
  format') with realistic 400 Bad Request tests: missing required fields
  and an invalid AnomalyType enum value.

- Add afterAll teardown guard: suppress expected MongoDB shutdown-race
  unhandled rejections (MongoClientClosedError, MongoWriteConcernError
  ShutdownInProgress, MongoServerError InterruptedAtShutdown) that the
  driver emits when the in-memory replica set is stopped.
…n metadata

Introduce ViolationMetadata schema and plumbing to store plain-English reasons, duration, consecutive frame counts, and signal strength with violation anomalies. This ensures that instructors reviewing violation logs can easily understand why a violation was flagged without needing to understand raw biometric geometry or detector internals.

This commit respects the 'trust, not surveillance' project philosophy by replacing opaque black-box flagging with structured, auditable, and human-readable metadata explanations.

PR 1:
- Add ViolationMetadata interface to backend Anomaly.ts and frontend reportanomaly.types.ts.
- Add optional metadata field to backend IAnomalyData class (persisted to DB in constructor).
- Add metadata field to NewAnomalyData validator DTO with class-transformer @Transform to parse incoming stringified JSON metadata in multipart requests.
- Add integration tests verifying successful metadata persistence and round-tripping through POST /anomalies/record/image.
This introduces real-time liveness detection (stillness, freeze-frame, looking away) built on top of the ViolationMetadata schema from PR 1.

This commit respects the 'trust, not surveillance' project philosophy by:
- Requiring a sustained window (12s stillness, 5s looking away) and a minimum consecutive frame count (8 frames) before flagging.
- Providing detailed plain-English violation descriptions for instructors (visible via tooltips).
- Allowing course coordinators to toggle this feature on/off per-course.

PR 2:
- Register livenessDetection proctoring toggle in EditProctoringModal, ISettingRepository, and AnomalyType enum.
- Enforce server-side settings gating for LIVENESS and LOOKING_AWAY anomalies in AnomalyController.
- Build LivenessDetector React component supporting 12s stillness drift, 15s freeze-frame/photo variance check, and 5s looking away detection.
- Implement debouncing (8 consecutive frames) and per-type cooldown (60s).
- Build central useViolationReporter hook to capture canvas frame and submit anomalies.
- Add teacher explanation badge with tooltip displaying full violation details on AnomaliesList table.
- Add unit tests for stillness, freeze-frame, and head-turn math in Anomaly.test.ts.
@github-actions github-actions Bot added frontend Changes to the frontend of the project backend labels Jul 13, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backend frontend Changes to the frontend of the project

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant