Skip to content

Conversions: runtime compliance guards (contact window, 7-in-7 counter, AMD) + LLM outcome classifier #11

Description

@teetangh

Context

Two gaps between the written policy (data/policy/v1.yaml: contact window 8am–9pm borrower-local, max 7 contacts per 7 days, required disclosures) and runtime behavior:

  1. Nothing enforces contact rules at dial time — the policy is only checked by the offline compliance evaluator (learning/compliance.py). A misconfigured cron or operator can dial at 6am; nothing counts contact frequency; voicemail pickups get the full negotiation script (an FDCPA third-party-disclosure risk).
  2. Outcome/conversion measurement is keyword-matching_binary_resolution_from_text (used by the eval loop) misses paraphrases, flagged in the repo's known limitations as fragile.

Implementation plan

Runtime compliance guards:

  1. apps/api/compliance_runtime.py: check_contact_allowed(borrower_id, tz) → verdict + reason. Time-window check from borrower timezone (add tz to borrower context; default conservative — only 11am–8pm UTC-overlap when unknown). Contact counter in Redis (INCR with 7-day expiry keyed contact:{borrower_id}:{date}), counting attempts not connections.
  2. Enforce at both dial-init endpoints (/voice/dial-init, /voice/streaming/dial-init in apps/api/voice_twiml.py) → 403 with machine-readable reason; Temporal resolve_via_voice activity respects the same check (retryable-later error).
  3. Voicemail/AMD: place outbound calls with Twilio Answering Machine Detection (machine_detection="DetectMessageEnd" on the call-create in voice_twiml.py:52); on answered_by=machine*: play a compliant identification-only message (no debt details — FDCPA third-party rule), hang up, record outcome no_contact.

Outcome classification:
4. Replace _binary_resolution_from_text keyword heuristic with an LLM-judge classifier using the existing judge role from settings.yaml (cross-family, already used by learning/compliance.py's layer 2): few-shot prompt over the final transcript segment → {deal_agreed|no_deal|hardship_referred|no_contact|abandoned} + confidence. For live streaming calls the end_call tool args stay authoritative; the classifier covers eval-loop simulations and TwiML-path transcripts.
5. Tests: window/counter unit tests (freeze time, fakeredis); AMD callback handling test; classifier evaluated against the 10 scripted scenarios' expected outcomes.

Acceptance criteria

  • Dial-init outside the allowed window or over the 7-in-7 limit → 403 with reason; Temporal path defers instead of dialing.
  • answered_by=machine: no negotiation content spoken, outcome no_contact recorded.
  • Classifier matches expected outcomes on all 10 scripted scenarios and disagrees with the keyword heuristic on at least the known paraphrase cases.

Dependencies

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium priorityconversionsNegotiation outcomes, conversions, and complianceenhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions