PinSieve Improves VLM Serving and Content Quality Triage in Production

Enterprise AI agents often require boundaries, statefulness, observability, and governability rather than full autonomy in production environments. A new system called PinSieve has been developed as a production case study within a large-scale content-quality pipeline. Its deployed component is a selective Vision-Language Model (VLM) Serving Agent that specifically targets the "grey-zone" slice of content that lightweight upstream models cannot resolve, providing a scalar routing score online and maintaining human escalation control. This deployed system has demonstrated substantial improvements, filtering 2.05 times more non-actionable items than its predecessor while slightly reducing the estimated miss rate. After its promotion, PinSieve boosted review productivity by 25.7%, cut normalized operating costs by 16.2%, and accelerated signal delivery from next-day to same-day. Maintenance of PinSieve is managed through a governed memory flywheel that uses selective feedback, where escalated items are reviewed by default and auto-passed items are primarily labeled via audit sampling. This feedback memory records routing traces, observation paths, audit propensities, and replay metadata for evaluation and debugging. A Data Curation Agent employs a bounded proposal-verifier loop over representative, uncertainty, recency, and fresh-review replay, with guardrails for positive-rate and score-bin before batch acceptance. This design reduced the average FNR@50% from 17.73% to 13.29% over six months of production data. A Reasoning Review Agent further audits teacher-generated rationales to support keep/repair/drop decisions. The serving-agent recipe has shown transferability to other internal signals.