webmcp/analysis

Pillbox

A caregiver and a pharmacist each get an agent on one medication list, every flag quoting the FDA label.

Aggregate 37.5
Leverage 10
Execution 8.5
Impact 10
Creativity 9

Each criterion 1–10, equally weighted; aggregate is their sum. Ranking is the pipeline's consolidated output.

01 Links & metadata

Category
health / health
Origin
built for the challenge / built for the challenge
Access
Eligibility
LIKELY_ELIGIBLE / LIKELY_ELIGIBLE
Substitution
TRANSFORMATIVE / TRANSFORMATIVE
Demo liveness
alive

Origin, access model, eligibility, and substitution are reviewer diagnostics, not judging criteria. Authentication requirements are not penalized.

02 The two blind reviews

Two independent reviewers scored this project blind, from a sanitized evidence packet. Scores are shown separately so the reasoning stays inspectable. A withheld score means the reviewers differed by more than two points.

Reviewer A (round 1)

confidence 82%
Leverage
9 /10

WebMCP provides role-scoped tools over shared state, making the disagreement between caregiver and pharmacist capabilities part of the product. Structured checks and source-linked evidence are difficult to reproduce reliably through generic UI driving.

Evidence cited
  • ABOUT TEXT describes 12 caregiver tools versus 9 pharmacist tools on the same origin/list
  • check_interactions runs before adding a new medication
  • Every flag includes verbatim FDA sentence, set_id, URL, and section-derived severity
  • accept_change and propose_change enforce distinct roles
Execution
8 /10

The packet gives detailed domain behavior, data provenance, permissions, and a concrete ciprofloxacin/warfarin scenario. Video exists but visual inspection failed, so the claimed demo is not directly observed.

Evidence cited
  • ABOUT TEXT reports 64 drugs, 1,012 interaction sentences, and 497 geriatric-use sentences tied to set IDs
  • Two rows turn red on the described medication addition
  • FACTS report public repo, live demo, and 119-second pillbox demo
Impact
10 /10

Caregivers and pharmacists face consequential medication reconciliation and interaction-checking risks; source-grounded, shared review can provide substantial safety value for a specific real audience.

Evidence cited
  • ABOUT TEXT gives a credible caregiver scenario involving warfarin and ciprofloxacin
  • Exact FDA label quotations and links address the weakness of unexplained severity labels
  • Role-separated shared list supports caregiver-pharmacist collaboration
Creativity
9 /10

The disagreement in role-specific WebMCP surfaces is an unusually strong interaction concept, and the insistence on verbatim label evidence shows depth beyond a generic medication app.

Evidence cited
  • Same page exposes different tool sets by role and makes that divergence the product
  • Label indexing across brands/classes/sections is described as a deliberate data solution
  • Agent collaboration is constrained by pharmacist/caregiver authority

Reviewer B (round 2)

confidence 92%
Leverage
9 /10

The product's central value is shared human-agent state with distinct tool permissions and auditable structured evidence; ordinary UI driving would be unreliable and would not naturally preserve this collaboration model.

Evidence cited
  • About text says caregiver and pharmacist links expose different WebMCP tool sets on the same list.
  • Frames show medication list, agent workflow, FDA/openFDA retrieval, and graph-like evidence paths.
  • The project explicitly keeps exact label sentences, set IDs, and source URLs attached to flags.
Execution
9 /10

The packet gives unusually detailed and consistent evidence of the core workflow, including list state, medication addition, interaction detection, role separation, and traceable source evidence; the 119-second video is represented by frames and packaging.

Evidence cited
  • Frames show the 11-medication dashboard and ciprofloxacin addition with warning state.
  • Frames show connected FDA/openFDA processing and caregiver-facing result UI.
  • Description specifies 64 drugs, 1,012 interaction sentences, and exact source linkage.
Impact
10 /10

Medication reconciliation for an older adult across caregivers, pharmacists, and multiple doctors is a high-stakes real problem; the demonstrated source-grounded warning directly addresses it.

Evidence cited
  • Narrative identifies an elderly patient, multiple prescriptions/doctors, remote caregiver, and missed warfarin/ciprofloxacin checking.
  • Frames visibly show a consolidated list and interaction-warning workflow.
Creativity
9 /10

Role-specific agents on one medication list plus verbatim-label evidence and disagreement as product behavior is an ambitious, memorable collaboration model.

Evidence cited
  • Different windows intentionally expose different tools and authority.
  • Safety flags are tied to exact FDA label text rather than opaque summaries.

03 Review highlights

Standouts across reviewers

  • Role-specific shared WebMCP capability is central, not decorative
  • FDA-label-grounded explanations with exact provenance
  • Exceptional source-grounded safety workflow and shared role-specific state.

Red flags

  • High-stakes medical claims are packet claims and need observed validation
  • Visual inspection of the demo frames failed due vision-service file-descriptor exhaustion

04 Evidence

What each artifact proves is labeled on the artifact itself. A Devpost page capture is packaging evidence, not proof the product runs; video frames are evidence from the submitted demo, not live verification. Probe captures come from Stage 2 interactive testing of the live product by a reviewer.

Devpost page capture for Pillbox
EX-01 Devpost page capture · packaging evidence, not runtime proof · source
Live probe of Pillbox before interaction
EX-02 Live probe at Stage 2 · observed product behavior, reviewer-driven
Live probe of Pillbox after interaction
EX-03 Live probe after interaction · observed product behavior

EX-V Submitted demo video — “pillbox demo”

Contact sheets from the 119s video the team submitted. This is what reviewers were shown; it demonstrates the product in motion but is not independent verification. · watch the original

Contact sheet 1 from the Pillbox demo video
EX-V1 Sheet 1 of 3 · reported video evidence
Contact sheet 2 from the Pillbox demo video
EX-V2 Sheet 2 of 3 · reported video evidence
Contact sheet 3 from the Pillbox demo video
EX-V3 Sheet 3 of 3 · reported video evidence