The Prove-It Pod

Revised after further material arrived. A 158-file judge pack arrived after the first pass: the seven-point submission document previously scored as absent, day-numbered prompt logs across five workstreams (Kyle's with kept-vs-redone notes, Etienne's a timestamped 220-prompt OpenCode extraction), 14 daily meeting minutes plus a consolidated decision register, a full token/cost accounting (~$1,158 of the EUR 1,500 cap), Jira worklogs, and a 203MB Playwright execution report (117 tests, 112 passed). This lifts documentation, process, ai_usage, efficiency, candour, and modestly testing and deployability, moving the total from 57 to 71. It does not change the code: POST /api/dispatch still has no authentication or approval check, and the pack sharpens that finding, because the pod's own decision register states 'Campaign approval and SMS dispatch require authentication' while no document in the pack discloses that the backend does not enforce it, and the committed secrets are likewise undisclosed. Acceptance, security, code, architecture and model_choice therefore stand.

What this artifact can and cannot show. Submitted as GitLab source exports, not repositories: no .git in any of the five archives, so commit history, author records and freeze tags are absent. Freeze compliance and journal cadence are therefore unverifiable from this artifact — recorded as unverified, never as violations. All 158 documentation files carry the same archive timestamp, so their write time cannot be established either. The other chartered pod's material carries per-page dates. That is a difference in what each artifact can show, not a demonstrated difference in how the pods worked.

POD1's NGAIGE is a FastAPI monolith (SQLite, LiteLLM in front of self-hosted vLLM, Twilio, Auth0) with two prebuilt React SPAs, an infra repo of RunPod/vLLM tooling, and a 520-file Playwright suite, now accompanied by a substantial judge pack: submission document, daily minutes and decision register, per-workstream prompt logs, token/cost accounting and a final test-execution report. The mechanisms for all six acceptance items exist in code and the API boots and runs offline in simulation mode. The decisive finding stands: the human-approval gate (item C) is enforced only in the frontend bundle; backend POST /dispatch has no approval check and, as proven live, dispatches with only an auto-issued session cookie and no login, even though the pod's own decision register asserts dispatch requires authentication. Sovereignty is respected (open-weight models throughout), but real-shaped secrets are committed across several tracked files and neither top-severity issue is disclosed in the pack. This assessment runs alongside the panel's six-axis charter rubric and decides nothing.

71 / 100 scored

Tokens and spend, as the pod reported them

Tokens reported	1,114,540,355
By member	Yanika 556M, Etienne 387M, Kyle 83M, Praveen 77M
Spend reported	$1,158.33
Models named	GLM-5.2 (devs) and Qwen3 (app), per the same document

Source: Tokens_Usage.docx — per-member table. These are the pod's own figures, not measurements by this assessment. Cap: €1,500 per pod.

The document states DevOps Claude Code usage is excluded from the token record, so the AI-usage picture is incomplete by the pod's own note.
Praveen appears as 76,875,200 in the member table and 162,612,236 as a subtotal elsewhere in the same document — a factor-of-two discrepancy the pod should reconcile.

Effort — stated against demonstrable

Hours stated	299.0 h
Source	worklog_by_week.csv (Jira export), 5 people, weeks 33-36
Per person	Yanika 77h, Duncan 70h, Kyle 65h, Praveen 57h, Etienne 30h
Above the 45 h/person allowance	Duncan +26h, Kyle +20h, Praveen +12h, Yanika +32h
Commits	no history in the artifact

Submitted as GitLab source exports with no .git, so no commit history, author records or timestamps exist. Nothing in the artifact can corroborate or contradict the stated hours.

The only pod that reported hours at all, and the figures exceed the charter's stated per-person allowance for four of five members. Reported openly in a Jira export the pod chose to hand over — a pod that hid its hours would show no exceedance at all.

A commit is not an hour and a commit count is not effort. This bounds when work happened, not how long it took, and no score is derived from it.

Prompt craft 8 / 10

The most complete prompt record of the trial, and most of it is verbatim. Five workstreams each left artifacts in their own working style: the FE series is day-numbered (Day 1-12) with real API schemas, polling rules and navigation contracts, so you can watch the mock spec harden into an integrated product; Kyle's backend log interleaves prompts with pasted stack traces and candid margin notes about what the model got wrong and what he replaced by hand, which is exactly the kept-vs-redone evidence the charter asks for; QA submitted agent definitions, skills and 7.2MB of raw OpenCode session JSON that confirms the work ran on the self-hosted GLM-5.2 proxy. The reusable core (the Day-1 product spec, reused verbatim by QA as test-plan input; the QA healer/planner agents) would transfer to another team. Points off because the 120KB Etienne dump is unfiltered session chatter that buries its good material, the DevOps file is paraphrase rather than verbatim, and coverage thins after ~24 Aug.

best — submissions/pod1-prove-it/docs/judge-pack/AI Prompts/Kyle's Prompts/Backend Prompts.txt
Kyle's note: ... it created the command-line directory and the `/api` prefix. But it generated a lot of slop for the middleware that didn't work. I replaced it manually with a working one-liner.
best — submissions/pod1-prove-it/docs/judge-pack/AI Prompts/FE Prompts/Day 10 - ClientApplicationPrompt - DispatchCampaigns.txt
While the campaign status is `IN_PROGRESS`, call GET `/dispatch` every 5 seconds ... Continue polling until the campaign reaches either: `COMPLETED` or `FAILED`
weakest — submissions/pod1-prove-it/docs/judge-pack/AI Prompts/Etienne Prompts/Prompts/prompts.md
hello pupu, what are you doing today?

Assessed separately from the twelve categories: the trial's question was how each pod worked with models, and the prompts are the record of that.

Flagged for a second look

Things a reader would want to know. Not findings against the submission and not scored — questions, oddities and loose ends.

Etienne's 120KB prompt dump is a raw extraction of all 31 OpenCode sessions on the box, including non-project material such as installing ClickHouse under /opt/applications and greetings like 'hello pupu' from 16 Aug, before some workstreams began.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/Etienne Prompts/Prompts/prompts.md
The Day-2 mock frontend prompt is filed as 'MockClientApplicationPrompt - GoogleAI', naming a third-party Google model for product UI generation, while the charter confines third-party AI to the platform layer; nothing inside the file explains the tool choice.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/FE Prompts/Day 2 - MockClientApplicationPrompt - GoogleAI.txt
The DevOps 'prompts' are third-person paraphrases ('A screenshot of the blank Teams message, with "for some reason the webhook is empty"') rather than verbatim prompts, unlike every other POD1 workstream, and both scripts they describe are RunPod cost-ops rather than product work.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/DevOps - Promts/DevOps Prompts.txt
Kyle's backoffice prompt explicitly instructs that 'any authenticated user should be allowed to access' the /api/backoffice endpoints, so the admin surface's flat authorization model was specified in the prompt, not introduced by the model.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/Kyle's Prompts/Backend Prompts.txt
Etienne's Readme records that a whole-backend-at-once trial silently 'reverted to previous requirements and implemented partial implementation', costing over two days before a re-plan recovered it, and that a richer registration flow was built and then deliberately cut as too complex.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/Etienne Prompts/Prompts/Readme.md
The raw QA session JSON confirms QA ran on the self-hosted stack (modelID haud-runpod-glm-5-2-proxy via the prove-it LiteLLM), but all six session files are from a single day, 24 Aug, so verbatim QA evidence covers one day of a 15-day trial.
submissions/pod1-prove-it/docs/judge-pack/AI Prompts/QA Prompts Skills and Agents/OpenCode Session 24-08/

Repository	GitLab source exports, 5 repos (no git history) + 158-file judge pack
Received	repos 31 Aug 11:20 CEST · documentation pack 31 Aug 12:03 CEST

Assessment by category

Acceptance coverage

partial4/9

Do items A–F actually work, on demand, without the pod driving?

Judge justification

Unchanged: the code is what it was. Mechanisms for A-F are present and the app runs, but the headline refusal property fails at the boundary: the AGREE gate lives only in the SPA (sms-campaign-mockup/src/pages/MessagesPage.tsx:123 canApprove), while backend POST /dispatch (ngaige/api/routes/dispatch.py:150) has no approval check and no auth dependency; I seeded a session with no user and dispatched a campaign. The new pack sharpens rather than softens this: Consolidated_Decision_Register.md section 2 states 'Campaign approval and SMS dispatch require authentication', a requirement the backend does not implement. Per-client Sender ID remains absent (now explicitly listed in section 7.4, 'No Sender ID registration or selection'), consent remains asserted-by-upload (register section 7: uploads 'treated as a consented recipient list'), and F still lacks inbound STOP-keyword capture.

What would make this better

Move the approval gate server-side: require an explicit approved-by-user flag and authentication on POST /dispatch, and reject dispatch when the campaign has not been approved by the session's logged-in user, which is what your own decision register already says the system requires.

What this category looks for

A onboard SME: business name, sender ID, consented recipient list
B campaign created conversationally through the Telegram POC
C engine drafts the plan and a human must approve before dispatch
D dispatch to 50+ test MSISDNs, delivery receipts ingested
E report of sent / delivered / failed / cost, reproducible by script
F STOP honoured, opted-out excluded from the NEXT send

C and F are refusal properties — the system must decline to do something. A demo can look perfect with the mechanism absent, so these are probed adversarially rather than asked about.

Architecture & technology

partial6/8

Are the technology choices ones you would defend in six months?

Judge justification

Unchanged in substance: a single-host modular-monolith FastAPI app serving two SPAs, LiteLLM as the AI boundary, background pollers for dispatch status and Telegram; defensible for a fifteen-day trial. The decision register now documents the rationale behind the load-bearing choices (Marketing Agent vs orchestrator separation, LiteLLM abstraction so the model can change without app changes, SQLite as a deliberate trial choice with the limitation stated), which confirms the choices were deliberate but does not change what was built.

What would make this better

Document the intended production hardening path for the two trial choices you already flag (SQLite to Postgres, and where role-based Client/Back-Office separation would attach) so the modular-monolith boundaries survive that migration.

What this category looks for

Component boundaries match the problem, not a framework's defaults
Dependencies justified; nothing heavy pulled in for a few lines
The orchestrator / engine split is real, not cosmetic
Failure paths considered: what happens when the gateway is down
Choices explained somewhere, not just made

Novelty is not a virtue here. Boring and correct beats clever and unexplained.

Code quality

partial6/8

Could a new engineer change this safely next week?

Judge justification

Unchanged: typed SQLAlchemy 2.0 async style, consistent router-per-resource layout, conventions in AGENTS.md; a new engineer could work here. Against that: _get_authenticated_user_id is copy-pasted across dispatch.py, stop.py and user.py; the opted-out sync block is duplicated in auth.py (twice) and contacts.py; errors surface via print() with no logging framework. The judge pack adds context on how the code was produced (screen-by-screen bounded prompts, manual fixes where the model produced 'slop') but no new code.

What would make this better

Lift the duplicated authenticated-user lookup and the opted-out-sync block into ngaige/api/dependencies.py so opt-out enforcement has one definition rather than three drifting copies.

What this category looks for

Readable without a guide; naming that matches the domain
No large copy-paste blocks or dead speculative abstraction
Errors handled where they occur, not swallowed
Consistent style across the codebase, including AI-written parts

Volume earns nothing. A small codebase that does the six things beats a large one that does four.

Testing & verification

partial5/8

Do the tests prove the acceptance items, or just exercise code?

Judge justification

Raised from 4. The pack supplies what the repos could not: a final Playwright HTML report (docs/judge-pack/Testing/latest-ngaige-test-report/, 203MB with traces, videos and AI failure analyses) matching the submission document's recorded run of 117 tests, 112 passed, 3 failed, 2 skipped (~95.7%), plus a tests HLD and a QA agent flow. That proves the suite really ran against the deployed app with an honest pass/fail record. The core critique stands: no test proves a refusal property at the API boundary; nothing asserts that unapproved or unauthenticated dispatch is rejected (it is not), and nothing opts a number out, dispatches again, and asserts exclusion. The suite still targets the live host and could not be run here.

What would make this better

Add a backend test that POSTs /dispatch on an unapproved campaign and asserts a 4xx, and one that opts a number out, generates messages, dispatches, and asserts that number received nothing.

What this category looks for

Tests exist, run from a clean clone, and pass
The refusal properties (C, F) have tests of their own
Verification evidence beyond unit tests: scripts, fixtures, logs
A failing test would actually catch a real regression

Coverage percentage is not read. A single test that proves STOP excludes on the next send is worth more than 80% line coverage.

Deployability & operations

partial6/8

Clean clone to running system, following only the written steps.

Judge justification

Raised from 5. The API still boots clean from its README (uv sync, uv run python main.py, migrations auto-run, simulation mode). The previously missing top-level runbook now exists in minimal form: 'Prove-it-Pod - Installation Guidelines.pdf' gives the full-stack path (envbuilder compose, Nexus registry, cert placement, 'docker compose --env-file .env.demo up -d traefik haud-ngaige-app'). It is one page, presumes access to their Nexus and GitLab, does not document environment variables beyond pointing at .env.demo, and could not be exercised from here, so it closes the gap on paper rather than demonstrably.

What would make this better

Expand the installation page with the required environment variables (LiteLLM endpoint, Auth0, Twilio) and a variant that does not depend on the private Nexus registry, so a HAUD engineer outside your network can stand the stack up.

What this category looks for

One documented command, or a short documented sequence
Configuration and secrets handled sanely; .env.example present
It actually starts — verified, not asserted
Teardown, logs and health checks exist

This is scored by doing it. The result is binary and unkind: either the written steps produced a running system or they did not.

Documentation & handover

solid7/8

Could HAUD take this over without the pod in the room?

Judge justification

Raised from 6. The seven-point submission document scored as absent now exists (NGAIGE_Delivery_Acceptance_Evidence_Assessment.md) with all seven sections, alongside the HLD, a tests HLD, infra info, installation guidelines and the decision register; the handover surface is now real. It is weakened by unfinished edges: section 3 lists AR-01..13 but never marks them Met/Partially Met/Not Met, section 5's reproduction guide has 'TBD' for test data, accounts and expected results, template text survives ('Complete this section from the submitted repository'), and the section 9 checklist ticks items the document does not actually contain (commit SHA, exact acceptance mapping).

What would make this better

Finish the acceptance mapping in section 3 (each AR marked Met/Partial/Not Met with the code path and reproduction step) and replace the TBD fields, so the document delivers what its own checklist claims.

What this category looks for

The submission .md answers all seven requested points
Claims cite where in the repo the implementation lives
Each claim says how to reproduce or test it
A runbook for operating it, not just a README for reading it

The bar is the client-bound mainline: would you hand this to someone who has never met the authors?

AI usage & prompt craft

solid7/9

The A/B payload — how the pod actually worked with models.

Judge justification

Raised from 4, the largest mover. The pack contains a genuine working record across five workstreams: FE prompts day-numbered 1-12 showing the screen-by-screen API-integration method, Kyle's backend log with explicit kept-vs-redone annotations ('it generated a lot of slop for the middleware... I replaced it manually with a working one-liner'), Etienne's timestamped OpenCode extraction (31 sessions, 220 prompts, extracted 28 Aug), DevOps prompt narratives including debugging iterations, QA agent/skill definitions with OpenCode session JSON exports, and per-member per-day token tables (1.2B tokens total). The method the decision register describes (bounded changes, review, manual fixes where cheaper than reprompting) is visible in the artifacts. Held below solid-plus by the explicit exclusion of DevOps Claude Code usage from the token document, which leaves the AI-usage record incomplete, and by the FE logs being curated final prompts rather than transcripts.

What would make this better

Include the Claude Code usage record (even as a summary of sessions and what they produced) and one or two full transcripts per workstream, so the record covers everything the pod used rather than the metered subset.

What this category looks for

Prompt library or logs present and legible
Which models, used where, and why those
Evidence of iteration: what was kept, what was redone
Token usage per model where recorded
Human review of model output visible in the history

This is the whole point of the trial. A pod that shipped well but cannot show how it used AI has answered the wrong question.

Process & decision evidence

solid7/8

Decision log and daily journal — kept as work, not written on the 30th.

Judge justification

Raised from 3, the category most changed. What was 'no journal, no decision log, no prompt log' is now 14 daily meeting minutes (11-28 Aug, AI-generated from meeting transcriptions per MeetingMinutesPrompt.txt, substantive: progress, blockers with owners, decisions, QA findings), a consolidated decision register separating decisions from proposals and open questions, day-numbered prompt logs, and Jira worklogs. This is not the exact rule-6 artifact: the kept/redone record is patchy outside Kyle's notes, impediments carry owners but not time-lost, and 'one thing we'd prompt differently' appears nowhere. Contemporaneity cannot be proven, since all 158 files are dated 31 Aug 10:02 (the zip time) and there is no git history, but internal dating (minutes dated per day, Etienne's extraction stamped 28 Aug) and the natural grain of the content read as records kept during the build and consolidated at the end.

What would make this better

Keep the rule-6 fields as a literal per-day table (drove / kept / redone-and-why / impediment with time lost / prompt-differently) alongside the minutes, and export it from a timestamped system so cadence is provable.

What this category looks for

Decisions recorded with their reasoning and their date
Journal entries across the working days, not backfilled in a burst
Impediments logged with category, owner and time lost
Rework recorded honestly, including what was thrown away

Entries dated across fifteen days read differently from fifteen entries committed on the last afternoon. The git history shows which.

Candour & self-assessment

solid7/9

The requested LLM assessment, and whether it tells the truth.

Judge justification

Raised from 6. The self-assessment surface is now broad and genuinely unflattering: section 7 lists limitations across nine areas, Issues.docx adds a bug register, the section 8 LLM assessment flags its own weak evidence (missing acceptance traceability, unreproducible steps), and the 20 Aug minutes record that prompt injection succeeded and a fake medical-claim campaign was accepted, which few teams would put in front of judges. Two things cap it: neither of the highest-severity findings is disclosed anywhere in the pack (the unauthenticated /dispatch endpoint and the committed credentials), and the decision register affirmatively asserts a property the code does not have ('Campaign approval and SMS dispatch require authentication'). Read together this is an intent-verification gap, the pod stating what it believed rather than what it checked, not concealment, but it means the self-assessment is honest about what they knew and wrong about what they didn't.

What would make this better

Verify the security-relevant claims in the decision register against the API before asserting them, and add the two missing findings (no server-side auth/approval on dispatch, committed credentials) to the known-issues register.

What this category looks for

The LLM-generated assessment is present
It names areas where evidence is weak or the requirement is unmet
Known issues, limitations and shortcuts are listed specifically
The self-assessment matches what the repo and probes actually show

A submission that names its own gaps is more trustworthy than one that claims six of six. Overclaiming found by probe is scored here, and it is the most damaging single finding a submission can carry.

Model choice & sovereignty

solid7/8

Which models, were they the right ones, and did they respect the boundary?

Judge justification

Unchanged: open-weight throughout and the sovereignty boundary holds; the product reaches models only via LiteLLM fronting self-hosted vLLM (Qwen3-VL-4B-Instruct for the app, GLM-5.2-W4A16 on RunPod for development). The pack's section 6 now documents this deliberately, and the register records real selection reasoning (vision routing, cost/startup/memory tradeoffs, refusing a late model switch as unnecessary risk) plus an honest sovereignty caveat (RunPod EU location is not legal sovereignty). The weakness stands: the app takes the first model LiteLLM lists rather than pinning per task.

What would make this better

Pin an explicit model per task (chat, contact extraction, SMS generation) instead of taking the first model LiteLLM returns, so a routing change on the proxy cannot silently swap the model your product depends on.

What this category looks for

Models named specifically, with versions — not 'an LLM'
Product code and trial data touched self-hosted open weights only
Third-party APIs, if used, stayed on the platform layer (rule 2)
Model choice fits the task: a frontier model where it earned its cost, a small one where it did not
The issued rig was actually used, not idled while work went elsewhere

Newest is not best. The scored question is fit and boundary, not leaderboard position. A pod that ran a mid-tier open model well scores above one that reached for a frontier API it was not permitted to use. A rule-2 breach is a finding here regardless of how good the output was.

Token & spend efficiency

solid7/9

What the budget bought — EUR 1,500 per pod, hard, every euro logged.

Judge justification

Raised from 4. The accounting the first pass could not close is now closed: Tokens_Usage.docx totals ~$1,158 (RunPod $1,138.33, Twilio $20) against the EUR 1,500 cap, sourced from the admin panel the pod built, with per-member per-day token tables (1.2B tokens, 1.19B prompt / 7.2M completion) and Jira worklogs by week and task. They knew where the money went. Two blemishes keep it from higher: DevOps Claude Code usage is explicitly excluded ('Which is excluded from this Doc'), so the record is complete in euros but not in usage, and the document contradicts itself on Praveen's total (76,875,200 in the member table vs 162,612,236 in the key-alias breakdown and day subtotals).

What would make this better

Reconcile the two Praveen totals and state the USD-to-EUR conversion against the cap, so the ledger is internally consistent and closes in the charter's currency.

What this category looks for

Token usage recorded per model, not guessed at the end
Spend logged as it happened, with the cap respected
Output per euro: what the money actually produced
Rented GPU hours matched to work — parked when idle, not left burning
Rework cost visible: what was regenerated and why

Underspending is not virtue and overspending is not vice; an exhausted budget is a workflow finding, not a failure. What is scored is whether the pod knew where the money went and can show it. A pod that spent the full cap and can account for it scores above one that spent half and cannot.

Security & data handling

token2/8

Consent, opt-out, secrets and the sovereignty boundary.

Judge justification

Unchanged: the pack contains no code changes and discloses neither issue. Real-shaped secrets remain committed in tracked files (Twilio auth token, Telegram bot token, Auth0 client secret in README and envbuilder compose, LiteLLM keys, VM SSH password in .gitlab-ci.yml, Postgres password, RunPod API key, HF tokens), and the register's decision to reuse 'the same Twilio API credentials... for the hosted audit instance' compounds the exposure. POST /dispatch still requires no authentication or approval despite the register asserting it does; consent is still asserted by upload; role separation is absent (this one is disclosed). The sovereignty boundary being respected is what keeps this off the floor. No credential was tested against a live service.

What would make this better

Rotate every committed credential, move them to runtime secrets outside the repo, and put an authentication dependency plus an approved-campaign check on POST /dispatch before anything can be sent.

What this category looks for

Consent recorded on the recipient list, not assumed
Opt-out state durable and honoured across campaigns
No credentials committed; secrets loaded from environment
Trial data synthetic; sovereignty boundary respected
Sender ID and MSISDN handling sane

A campaign platform that leaks a recipient list or ignores an opt-out is not a product, whatever else it does well.

Evidence

Category	Item	Where	Note
acceptance	AGREE gate exists only in the SPA	`repo/sms-campaign-mockup/src/pages/MessagesPage.tsx:123`	canApprove = agree.trim().toUpperCase() === 'AGREE' && hasEnoughBalance; this is the only place AGREE is enforced. The string 'AGREE to unlock Approve' is also baked into the shipped bundle repo/ngaige-api/frontend/dist/assets/index-Cox8hJco.js.
acceptance	Backend dispatch has no approval or auth check	`repo/ngaige-api/ngaige/api/routes/dispatch.py:150`	dispatch_campaign_messages depends only on get_db and get_active_chat. No approval flag, no get_authenticated_user. I booted the app, seeded a session with user_id NULL, POSTed /api/dispatch with just the session cookie, and it returned {campaign_id, sent_count:1}. Re-confirmed against the same source after the judge pack arrived.
candour	Decision register asserts dispatch requires authentication; code does not enforce it	`docs/judge-pack/Decision Logs - Meeting Minutes/Consolidated_Decision_Register.md (section 2)`	'Campaign approval and SMS dispatch require authentication, although the trial allowed users to progress through much of the creation journey before logging in.' The backend enforces no such requirement on POST /dispatch, and no document in the pack (section 7.1 known issues, Issues.docx) discloses the gap. Adjacent auth gaps (shared Auth0 tenant, no role separation, no email verification) are disclosed.
security	Committed live-shaped secrets, undisclosed in the pack	`repo/ngaige-api/envbuilder/docker-compose.yml`	NGAIGE_TWILIO_AUTH_TOKEN, NGAIGE_TELEGRAM_TOKEN, NGAIGE_OAUTH_CLIENT_SECRET inline; Auth0 secret also in README.md; VM_PASSWORD in .gitlab-ci.yml; LiteLLM key, 3x HF tokens and a Postgres password in ngaige-infra. Not mentioned anywhere in the 158-file judge pack; the register (section 12) additionally records reusing the same Twilio credentials for the audit instance. Presence and shape only; nothing tested against a live service.
documentation	Seven-point submission document now exists, with unfinished edges	`docs/judge-pack/NGAIGE_Delivery_Acceptance_Evidence_Assessment.md`	All seven sections present (delivered, acceptance mapping, evidence index, reproduction, models, known issues, LLM assessment). Section 3 lists AR-01..13 without Met/Not-Met marks; section 5 reproduction has TBD test data/accounts; section 9 checklist ticks items the document does not contain (commit SHA).
ai_usage	Prompt logs with kept-vs-redone annotations	`docs/judge-pack/AI Prompts/Kyle's Prompts/Backend Prompts.txt`	53KB backend prompt log with inline outcome notes, e.g. 'it generated a lot of slop for the middleware that didn't work. I replaced it manually with a working one-liner.' FE prompts are day-numbered 1-12; Etienne's prompts.md is a timestamped OpenCode extraction (31 sessions, 220 prompts, extracted 2026-08-28); QA folder includes agent definitions and OpenCode session JSON exports.
efficiency	Full spend and token accounting, with one exclusion and one inconsistency	`docs/judge-pack/Tokens_Usage.docx`	~$1,158 total (RunPod $1,138.33 + Twilio $20) vs EUR 1,500 cap; per-member per-day token tables totalling 1,200,277,391 tokens. Notes 'DevOps also had access to Claude Code, Which is excluded from this Doc.' Praveen's total is 76,875,200 in the member table but 162,612,236 in the key-alias breakdown and day subtotals.
process	Daily meeting minutes with real QA findings	`docs/judge-pack/Decision Logs - Meeting Minutes/MeetingMinutes- 20.08.26.docx`	Day 8 minutes record per-person progress, blockers with owners (mobile-testing VPN problem, BlueStacks workaround), and adversarial LLM findings: prompt injection succeeded, a fake medical-claim campaign was accepted, a phishing campaign was refused. Fourteen dated minutes span 11-28 Aug; MeetingMinutesPrompt.txt shows they were AI-generated from meeting transcriptions.
testing	Final Playwright execution report supplied	`docs/judge-pack/Testing/latest-ngaige-test-report/`	203MB Playwright HTML report with traces, videos, screenshots and AI failure-analysis attachments, matching the recorded final run of 117 tests / 112 passed / 3 failed / 2 skipped (~95.7%) stated in the submission document section 2.7. Run against the deployed host; not reproducible from here.
model_choice	Open-weight models behind LiteLLM/vLLM, now documented	`repo/ngaige-infra/docker-compose.yaml:177`	vLLM serves Qwen/Qwen3.6-35B-A3B, Qwen/Qwen3-VL-4B-Instruct and zai-org/GLM-5.2; product reaches models only via LiteLLM. Judge-pack section 6 documents GLM-5.2-W4A16 (4xH200 on RunPod) as the dev model and Qwen3-VL-4B as the app model; the register records vision-routing and cost/availability reasoning.
deployability	One-page full-stack installation guide	`docs/judge-pack/Prove-it-Pod - Installation Guidelines.pdf`	Points at envbuilder, requires their Nexus registry (insecure-registries), cert placement under /opt/DOCKER_DATA/TRAEFIK, and gives the startup command 'docker compose --env-file .env.demo up -d traefik haud-ngaige-app'. Closes the missing-runbook gap on paper; not exercisable without access to their network.
deployability	API boots clean from README steps	`repo/ngaige-api/README.md`	uv sync then uv run python main.py started the app, auto-ran Alembic migrations, and served / (200) and /api/ping in simulation mode. Backend pytest: 11 passed.

Questions for the pod

Raised by the assessment, not answered by it. Each is a thing a reader of the submission would reasonably ask.

Your decision register states 'Campaign approval and SMS dispatch require authentication', but POST /api/dispatch has no authentication dependency and no approval check, and I dispatched a campaign with an anonymous session cookie. Was this requirement ever verified at the API boundary, and what stops an unapproved or unauthenticated dispatch?
The acceptance list frames F as the STOP keyword being honoured. The system models opt-out as an authenticated operator/list API, not capture of an inbound 'STOP' SMS reply. Was inbound-keyword capture in scope, and if so where is the Twilio inbound webhook?
The committed Twilio, Telegram, Auth0, LiteLLM, VM SSH and RunPod credentials, were these throwaway trial values already rotated, or live? The register notes the same Twilio credentials would be reused for the hosted audit instance.
Tokens_Usage.docx excludes DevOps Claude Code usage. Can that usage be summarised (sessions, what was produced) so the AI-usage record is complete, and which of the two conflicting Praveen totals (76,875,200 vs 162,612,236) is correct?
Item A asks for per-client Sender ID selection, which section 7.4 confirms is absent. Was per-client Sender ID descoped deliberately for the trial?

Could not verify

Checks that could not run. Recorded as unverified, never counted as failures.

Freeze compliance (rule 4) and the write-time of every judge-pack file: the repos are GitLab exports with no .git, and all 158 judge-pack files are dated 31 Aug 10:02 because that is when the archive was made. Internal dating (minutes dated per day 11-28 Aug, Etienne's prompt extraction stamped 2026-08-28) is consistent with contemporaneous records but is content, not metadata; recorded as unverifiable, not as a violation.
The final Playwright run (117/112/3/2): the supplied HTML report matches the claim and carries traces and videos, but it was produced against their hosted environment and could not be re-run or independently dated from here.
Item B (Telegram POC) end to end: the submission document itself says the demo was done on a phone with no recording ('N/A as it was done on the phone'); the TelegramWatcher flow was read in code but not exercised.
Item D (dispatch to 50 MSISDNs + receipt ingestion) end to end: no Twilio credentials and no sandbox gateway; the register and minutes report a live SMS trial but it could not be reproduced.
The excluded DevOps Claude Code usage: its volume and content are unknown; only its existence is stated in Tokens_Usage.docx.
SharePoint and GitLab evidence links in the submission document (pipelines, Nexus artifacts, hosted apps): all point at HAUD-internal or access-controlled resources unreachable from the review environment.
Whether the committed credentials are live: assessed for presence and shape only; none was tested against an external service.