A Practical Guide to Performance Testing for Enterprise Systems

Companies are rapidly integrating generative AI into procurement and sourcing decisions. The promise is obvious. AI can read thousands of pages of supplier proposals faster than any human team, summarize technical requirements in seconds, and create the appearance of consistency and objectivity in evaluation. But there is an important distinction managers are starting to overlook. The same AI system that performs extremely well at identifying whether a supplier meets minimum requirements may perform much less reliably when judging which supplier is truly better. Recent research published in the Journal of Business Logistics examined this issue by comparing how large language models evaluated supplier bids against evaluations completed by experienced procurement professionals. It analyzed 123 supplier proposals tied to 31 public procurement projects conducted by the State of Ohio between 2023 and 2024. The projects involved complex IT services contracts, many containing large, text-heavy bid packages requiring evaluative judgment rather than simple arithmetic comparisons. The researchers tested three reasoning-oriented AI models: OpenAI o3, Grok-3-Mini, and DeepSeek R1. They then compared their evaluations against human procurement scores. The findings revealed a surprisingly clear pattern. AI performed well when evaluating compliance signals. These are signals tied to baseline qualifications and technical requirements. Does…

Supply Chain Now Founder and CEO Scott Luton announced that its “Veteran Voices” podcast will re-launch this year in partnership with two non-profit organizations: The Guam Human Rights Initiative and the Military Women’s Collective. “We’re proud to promote the efforts of these two organizations, as we are all highly aligned in our dedicated mission to serve our veteran community. Both organizations are veteran-led, start-up nonprofits that are doing big things out in the market. They are the perfect partners for ‘Veteran Voices,’” said Luton. As noted on its website, the Guam Human Rights Initiative “is a collaborative research nonprofit seeking to advance human rights on Guam and throughout the Pacific through meaningful and impactful research efforts.” Since its founding in 2021, the Military Women’s Collective is determined “to make an impact in the military nonprofit space by focusing on our women veteran population.” Its mission focuses on empowering homeless and food-insecure women veterans as well as providing coaching, mentorship and support for women veterans who are transitioning out of the military. Mary Kate Soliva, host of “Veteran Voices,” said that she has a special place in her heart for both organizations and the good works they do. She is excited…