Start with one workflow that costs the most manual time. Prove value there before expanding.
Step 1: Ask what problem they’ve solved before, not what their tool can do
| Question area | Green flag | Red flag |
|---|---|---|
| References | Similar industry clients | Only generic demos |
| Data handling | Clear privacy policy | Vague on storage location |
| Pricing | Fixed scope option | Open-ended hourly only |
| Support | Named post-launch owner | Handoff to ticket queue |
Every vendor can describe their tool’s features. Far fewer can describe a specific business, in a situation similar to yours, where their solution actually solved the problem. Ask for that story in detail: what the business looked like before, what changed, and how they measured it.
Notice how specific the answer is. A vendor who genuinely solved a similar problem before will give you names, numbers, and a clear before-and-after. A vendor stretching to make an unrelated case study fit will speak in generalities and change the subject when pressed for specifics.
It is worth asking this question more than once, in slightly different ways, over the course of your conversations with a vendor. Consistency in the details from one conversation to the next is its own kind of evidence.
Reluctance here, more than any technical detail, is the clearest warning sign in the entire evaluation process.
Step 2: Ask to see it work with data like yours
A demo using the vendor’s own polished sample data tells you very little. Ask to see the tool run against something closer to your actual data, even a small, sanitized sample. This is where you find out whether the tool handles your real-world messiness or only performs well in a controlled demo.
A vendor confident in their product will welcome this request, since it is the fastest way for them to prove the tool actually works for you. Reluctance here, more than any technical detail, is the clearest warning sign in the entire evaluation process.
If a genuine data sample is not possible for privacy reasons, ask the vendor to walk through a scenario that matches your specific situation in detail rather than a generic example that could describe almost any business.
Working on something similar?
Let's talk →Step 3: Understand what happens to your data
Working on something similar?
Let's talk →Ask directly where your data is stored, who at the vendor can access it, whether it is used to train models that other customers benefit from, and what happens to it if you stop being a customer. You do not need technical knowledge to ask these questions, and you deserve a clear, direct answer to every one of them.
It also helps to ask what would happen to your data if the vendor itself were acquired or shut down, since that scenario is more common in this industry than most buyers expect, and a vendor with a clear answer has clearly thought about it before.
If the answer changes depending on who you ask, or if it is buried in dense legal language nobody on the sales team can explain plainly, treat that as a real concern rather than a minor formality. A vendor who handles customer data responsibly should be able to explain their approach to a non-technical person clearly and consistently.
Step 4: Get clear on what happens after the contract is signed
Ask who supports you after launch, how quickly they respond when something goes wrong, and what is included versus billed separately. A vendor’s true character often shows more clearly in how they describe ongoing support than in how they pitch the sale.
Ask this question even when the sales conversation feels like it is going well. The support conversation is the one most first-time buyers skip, and it is exactly the one that determines how the relationship actually feels six months after launch, well after the excitement of the initial pitch has worn off.
Your pre-decision checklist
Before you move forward, confirm:
- You have asked for a specific past example similar to your own situation.
- You have seen the tool tested against data resembling yours, not just a polished demo.
- You have a clear, direct answer about where your data goes and who can access it.
- You understand what support looks like after the contract is signed.
- You have spoken with at least two or three vendors, not just the first one you met.
- You have checked a reference client whose problem was similar to yours.
- You have confirmed how pricing changes if your volume or requirements grow.
