Case 01 · Tier-3 Hospital AW36-72B Private Deployment · 300+ Daily-Active Doctors · Guideline Lookup 8-15 min → 45s

TL;DR: Tier-3 Hospital AW36-72B Private Deployment · 300+ Daily-Active Doctors · Guideline Lookup 8-15 min → 45s. Direction: Private LLM Deployment. Scale: 300+ daily-active doctors. Hard KPI: clinical guideline lookup 8-15 min → 45s. Tech: AW36-72B, 4×A800, vLLM, Milvus, bge-large, BGE-reranker. My role: architecture, hardware selection, vLLM productionization, network integration, load testing, SLA. Boundary: PHI stays on-prem, no public egress, internal audit. Client name anonymized under NDA.

Problem

Doctors spend 8-15 minutes on average looking up clinical guidelines / drug interactions / charting norms — eating into ward rounds and case-discussion time.

Solution

AW36-72B on 4×A800, vLLM inference, Milvus indexes 500+ clinical guidelines + intramural norms, bge-large embedding + BGE-reranker.

Tech Stack

AW36-72B, 4×A800, vLLM, Milvus, bge-large, BGE-reranker

Technical boundary: PHI stays on-prem, no public egress, internal audit

Result (Hard KPIs)

clinical guideline lookup 8-15 min → 45s

My Role

architecture, hardware selection, vLLM productionization, network integration, load testing, SLA

Next: Download PDF · 30-min alignment

Client name anonymized · KPIs verified · Real client name disclosable under NDA.