Jerai Robins

Companies are rapidly integrating generative AI into procurement and sourcing decisions. The promise is obvious. AI can read thousands of pages of supplier proposals faster than any human team, summarize technical requirements in seconds, and create the appearance of consistency and objectivity in evaluation. But there is an important distinction managers are starting to overlook. The same AI system that performs extremely well at identifying whether a supplier meets minimum requirements may perform much less reliably when judging which supplier is truly better. Recent research published in the Journal of Business Logistics examined this issue by comparing how large language models evaluated supplier bids against evaluations completed by experienced procurement professionals. It analyzed 123 supplier proposals tied to 31 public procurement projects conducted by the State of Ohio between 2023 and 2024. The projects involved complex IT services contracts, many containing large, text-heavy bid packages requiring evaluative judgment rather than simple arithmetic comparisons. The researchers tested three reasoning-oriented AI models: OpenAI o3, Grok-3-Mini, and DeepSeek R1. They then compared their evaluations against human procurement scores. The findings revealed a surprisingly clear pattern. AI performed well when evaluating compliance signals. These are signals tied to baseline qualifications and technical requirements. Does…

Explore this Microsoft article on global supply chain innovation and resilience, featuring leaders who have shared their insights on our Supply Chain Now podcast! These influential guests joined us on recent Microsoft-sponsored shows, offering valuable perspectives on the future of supply chains. Read the full article here