The five tests.
Off-the-shelf AI is the right answer when any of the following five conditions are true. Custom commission is the right answer when all five are false.
Test 1: does an off-the-shelf product cover 80%+ of your need?
If a product like Karbon, Harvey, Spellbook, ServiceTitan AI, Vertafore IQ, Epicor Prism covers 80%+ of the workflow you want automated, off-the-shelf is right. The 20% gap is rarely worth a custom build. Tune the off-the-shelf product, accept the boundary.
Custom commission only makes sense if coverage is below 60%. Between 60% and 80% is a judgment call; we usually recommend off-the-shelf in this range.
Test 2: are your needs essentially the same as the average customer in your segment?
If your business is the average law firm, the average CPA firm, the average insurance agency, off-the-shelf is right. The product was built for you. If your business is meaningfully differentiated (specific matter taxonomy, specific carrier pool, specific part library, specific dispatch logic), custom is right.
Honest test: if a competitor visited your business and described what makes you different, would they describe specifics that off-the-shelf can't capture? If yes, custom. If no, off-the-shelf.
Test 3: is configuration substantive enough that the vendor's product becomes your business's product?
Some products allow deep configuration (Epicor Kinetic, ServiceTitan, Salesforce). Configuration done well closes most of the custom-vs-off-the-shelf gap. Other products are essentially fixed (most SaaS B2B products in the AI category). Configuration matters; ask the vendor honestly how much.
If configuration is real and your team can drive it, off-the-shelf wins. If configuration is shallow, custom wins.
Test 4: will the system stay relatively stable once shipped?
Custom commissions are scoped for systems that need 5-15% modification per year, not 50%. If the workflow is going to evolve materially every year, off-the-shelf vendors invest in the evolution; custom systems require the operation to invest separately each time.
If your workflow is a stable target, custom wins. If it is a moving target, off-the-shelf wins.
Test 5: do you want to own the system at handoff?
Off-the-shelf systems are rented; you stop paying, the system stops working. Custom commissions are owned; the operation holds the code, the data, and the deployment. For most workflows, ownership is a nice-to-have; for some (regulatory exposure, security-sensitive data, strategic differentiation), it is a must-have.
If ownership is a must-have, custom wins. If it's a nice-to-have, off-the-shelf is fine.
The honest summary.
Most workflows in mid-market operators pass at least one of the five tests on the off-the-shelf side. Most operators that come to us, after running the tests, end up commissioning for one specific workflow (the one that fails all five) and using off-the-shelf for everything else. This is the right pattern; trying to commission everything is a misallocation, and trying to off-the-shelf everything leaves the high-leverage commission on the table.
We use this framework on every diagnosis call. The answer is rarely "all custom" and rarely "all off-the-shelf." The answer is usually "this one workflow, custom; that one, off-the-shelf; this third one, hire a person."
How to apply.
Run the five tests on the workflow you most want automated. If 1-3 come back yes, look at off-the-shelf options first. If 4-5 come back no, the playbook library describes what we'd commission. If you want our read on which test your workflow fails, the diagnosis call is the next step.