I have sat in umpteen boardrooms where enthusiastic marketing directors present the results of their latest pilot. The numbers always look green, the feedback is glowing and the board nods in agreement. Yet, six months into the full launch, the wheels fall off. The results flatline and everyone is left scratching their heads.
The issue isn’t usually the product or the team. The problem lies with the marketing pilot schemes themselves.
We treat pilots as a safety net, a way to test the water before jumping in. However, in my experience working with scale-ups across the UK, these schemes often do more harm than good. They consume a disproportionate amount of effort and, crucially, they create a false environment that rarely reflects reality.
The Mirage of the False Environment
The primary reason marketing pilot schemes are misleading is that they exist in a bubble. When you run a pilot, you naturally throw your best resources at it. Your best salespeople are on the phones, your most attentive account managers are handling the clients and your marketing team is watching every metric like a hawk.
This intensity creates what sociologists call the Hawthorne effect; people behave differently because they know they are being watched. In a pilot, everyone tries harder. You might manually fix issues that would be automated (and break) at scale. You might offer a level of customer service that is simply unsustainable when you have ten thousand customers instead of ten.
The data you get from this environment isn’t wrong, but it is deeply misleading. It tells you what happens when everything is perfect. It does not tell you what happens on a rainy Tuesday in November when your systems are stressed and your team is tired.
The Hidden Cost of Effort
We also need to talk about the sheer effort required to set these up. Building a pilot often takes nearly as much energy as a real launch. You still need the creative assets, the tracking infrastructure, the legal sign-off and the budget approval.
By the time you have built the “test” version, you have often spent 80% of the resources needed for the real thing. If the pilot is successful, you then have to do the work all over again to operationalise it. If it “fails”, you have burned significant time on something that might have worked if it hadn’t been suffocated by the constraints of a test environment.
Why Do Marketers Persist with Pilots?
If marketing pilot schemes are so flawed, why do we keep using them?
Fear is the main driver. No one wants to be the person who signed off on a million-pound disaster. A pilot feels like a responsible, risk-mitigating step. It offers the illusion of certainty in an uncertain world. It allows stakeholders to say “we tested this” if things go wrong later.
There is also the lure of vanity metrics. Pilots are easy to manipulate. You can pick a friendly segment of your database, offer an aggressive discount that you can’t sustain long-term and watch the conversion rates soar. It looks good on a slide deck, but it isn’t a business strategy.
The Alternative: The Phased Rollout
So, if pilots aren’t the right thing, what is the alternative? In my opinion, you should stop testing and start launching.
I advocate for the phased rollout or “soft launch”.
The difference is subtle but critical. A pilot is a test: “If this works, we might do it.” A rollout is a commitment: “We are doing this, but we are starting here.”
- Real Systems: In a rollout, you use the actual systems and processes you intend to use at scale. No manual workarounds.
- Real mindset: The team knows this is the new reality, not a temporary project. They invest in fixing problems properly rather than patching them up for the duration of the test.
- Segmented Action: You launch to 5% of your market, or one specific geographic region (like the North West), or one vertical.
If it breaks, you fix it live. The data you get is real. The customers are real. The revenue is real. You aren’t simulating business; you are doing business.
Stop wasting time on false environments. Build the thing, launch it small and scale what actually works.
References and Further Reading
[1] Why 95% of AI Pilots Fail. https://www.vasco-consult.com/en/blog/01-12-2025-why-95-of-ai-pilots-fail-and-how-to-join-the-5-that-dont/
[2] Pilot Testing: How to Avoid Costly Release Mistakes. https://testsigma.com/blog/pilot-testing/
[3] Soft launch: Smoothly run the pilot of the loyalty program. https://www.openloyalty.io/insider/soft-launch
[4] 5 Reasons to Kill (or Rethink) a Pilot Project. https://www.riministreet.com/blog/5-reasons-to-kill-or-rethink-a-pilot-project/
Leave a Reply