Intelligent Supplier Contract Anomaly Detection

How the system actually runs.

Each stage below is a real component in the deployed pipeline. Stages outlined in teal are control points: the places where the architecture constrains what the model can receive or emit.

What it runs on.

How it is wired.

1. Fully cloud-native on Azure with no on-premises footprint.
2. Incoming supplier documents trigger an Azure Function, and Azure Document Intelligence extracts structured paragraph-level content.
3. Extracted chunks are embedded and stored in Azure AI Search alongside embeddings of the relevant contract sections.
4. The RAG pipeline retrieves the top matching contract clauses for each document chunk and passes them to GPT-4o for clause-level comparison.
5. GPT-4o output is constrained to a strict JSON schema: anomaly type, severity, affected clause reference, and plain-English explanation.
6. Procurement reviewers act on the structured anomaly report before approving any supplier payment.
7. Supplier documents and contract data live in entirely separate Azure Storage accounts with distinct access controls.

What keeps it inside its contract.

Guardrails were designed in from the start, not added after the fact. Where the system touches untrusted input, a control sits between that input and the model.

• Supplier document content is chunked at the paragraph level and passed through a semantic role classifier before the main analysis. Raw supplier-controlled text never enters the instruction context directly.
• GPT-4o output is constrained to a strict JSON schema, so the model cannot produce a free-form “clean bill of health” that bypasses the structured review gate.
• Red-team exercises specifically tested supplier-embedded instruction text designed to make the model output a false “no anomalies found” result. The paragraph classifier step mitigates it.
• A data processing addendum was negotiated with Microsoft, with a zero-retention policy on all Azure OpenAI API calls confirmed in writing.
• All Azure OpenAI traffic routes through private endpoints. No supplier data transits the public internet at any stage.
• Separate storage accounts with distinct access controls ensure a supplier document storage breach cannot expose the contract database.

What we still track.

No production AI system is finished. These are the live risks carried by this architecture, documented for the operating team rather than buried.

• A sophisticated supplier could embed adversarially crafted paragraph content to attempt model confusion at the clause-comparison stage if the role classifier has coverage gaps.
• Contract language ambiguity, such as rebate tiers with overlapping conditions, may produce false-positive anomaly flags that generate procurement review overhead without a real violation.
• Embedding drift as contract language evolves may reduce retrieval accuracy on older stored embeddings without a scheduled re-indexing job.
• Procurement reviewers may develop over-trust in anomaly reports and reduce independent scrutiny of complex multi-clause violation patterns.

What changed.

• Invoice pricing violations that previously passed through at scale became systematically detectable across the full supplier base.
• The procurement team shifted from reactive auditing after payment to proactive review of flagged anomalies before payment approval.
• Schema-constrained output prevented the model from producing misleading “safe” conclusions, and made every finding traceable to a specific contract clause.
• The zero-retention DPA gave the legal team a clear contractual basis for using Azure OpenAI with sensitive contract and supplier pricing data.

On numbers. Client identity and any identifying detail are withheld under NDA. Outcomes are stated as direction and materiality rather than as figures we are not free to publish.