Hype comes in cycles.
First it was NFTs. Then the metaverse. Then Web3. Every time a new technology explodes, agencies flood the zone with big claims, flashy decks, and “strategic innovation initiatives” that promise to future-proof your brand.
Don't even get us started on augmented reality.
Now it’s AI. Artificial Intelligence is everywhere right now. And most agencies are trying to cash in
We're not here to throw shade, AI is the real deal, transformative, exciting, and, yes, disruptive. But like every other major wave, it’s also become a magnet for vaporware, vanity projects and an absolute crap load of snake oil.
At XDS, we’ve always been skeptical of hype. Not because we’re tech-averse, but because we believe in solving real problems, not selling shiny objects. And right now, a lot of agencies are doing the latter.
So if you’re getting pitched an AI solution (and let’s be honest, you are), here are five questions that can help separate substance from snake oil:
1. What problem is this actually solving?
Before you even get into the tech, ask this: What’s broken? What’s inefficient? What’s painful for my users, customers, or my team?
We’ve seen “AI” tools pitched for everything from customer service to copywriting to lead scoring to MLR submissions. Some of those ideas make sense. Others are just… a hammer in search of a nail.
Smart agencies start with pain points, not platforms. If your partner is leading with capabilities instead of needs, that’s a red flag.
2. Is this real AI—or just a ChatGPT/API wrapper with a shiny UI?
We’ve audited multiple “AI-powered” tools that were essentially prefilled prompt templates behind a login screen and a fancy landing page.
That’s not to say those tools have zero value, but they’re not innovative. They’re not robust. And they’re not worth enterprise-level investment unless they’re connected to a broader strategy.
Ask:
- What model powers this?
- Is it fine-tuned? On what data?
- What happens when the API limits change or usage costs spike?
- How does this adapt to model enhancements and innovation?
If you get blank stares or buzzword salad, walk away.
3. Who owns it and maintains it long-term?
AI projects aren’t set-it-and-forget-it. They’re living systems. They need ongoing:
- Data input
- Quality control
- Prompt engineering
- Model updates
- Security checks
- Compliance oversight
If an agency can’t speak confidently about long-term governance or change management, you’re buying a short-term toy, not a scalable solution.
4. How will this fit into your actual workflow
We recently worked with a client who was pitched an “AI onboarding assistant.” The idea sounded good—but it had no integration with their HRIS, no link to their knowledge base, and required new logins for every team member.
It would have added friction, not reduced it.
True AI value comes from embedding into existing systems, not creating more systems to manage.
Ask:
- Will this integrate with our tech stack?
- Will it be invisible to users or disruptive?
- Who’s responsible for adoption?
5. Can we pilot this quickly and measure results?
Great ideas deserve a proving ground. The best AI partners will say: Let’s build a version, roll it out, and measure what happens.
Bad ones will keep you in “strategy phase” until the budget runs dry.
Push for:
- A test group
- Measurable KPIs
- Clear timelines
- A go/no-go decision point
The takeaway: Smart AI is quiet, strategic, and user-centric
The best AI projects aren’t loud. They don’t rely on jargon. They quietly transform workflows, simplify operations, and make your people more effective.
So before you say yes to the latest “AI-powered experience platform" or "AI-powered marketing strategy," take a breath. Ask better questions. And work with partners who’ve seen this movie before, and know how to rewrite the script.
Stay vigilant out there.
About the Experience Design Studio
The Experience Design Studio is an award-winning digitally native customer experience agency founded in 2017 by two agency veterans, bringing their collective creative, user experience, marketing, technology, and healthcare expertise together.
XDS is a full-service digital agency providing strategy to creation, consulting, design, engineering, marketing, and analytics, with the aim of providing seamless DTC and B2B experiences across all digital touch points, with common sense sprinkled in.