Experiments become systems that get used.
One great AI result is interesting. A system that can repeat it is valuable.
I experiment with AI on real work to discover what consistently improves the outcome, where it breaks and what makes the difference.
What holds up through repeated testing becomes a practical playbook or operating system. Turning a promising result into a repeatable way of working.
A demo shows what AI can do once. An experiment shows what you can rely on.
A polished output proves very little. To build a repeatable system, you need to know what caused the improvement, where it breaks and which parts hold up across repeated runs.
Every experiment here follows four steps. From a real task to a documented way of working.
The task
A recognisable piece of work with a reason to improve it.
The test
An AI approach compared with a credible way of doing the work today, across enough runs to reveal a pattern.
The evidence
Measured differences in time, quality or output, including failures, limits and the conditions that affected the result.
The system
Keep and document what holds up. Turn it into a reusable brief, check, workflow, playbook or operating system.
The long prompt found the right problem but funded the wrong fix.
I gave Claude Opus 5 one real business problem: £15,000, six weeks, fix one thing. Two prompts, three goes each.
Length wasn’t the lesson. Control was.
It found the problem losing £43,497 a month, then recommended fixing one losing £878.
Telling it how to do the job before it has seen the data.
It got no steps, went to the data first, and recommended fixing the £43,497 problem.
Give it the evidence and the decision to make. Let it work out the middle.
One task · one model · one day · six tests · scored by me
Everything in the hub, organised by what it teaches you.
The model will change. The work will still need to be done, so nothing here is filed under the AI product that happened to be used.
The Outcome Brief
Define the outcome, reader, source, constraints and quality checks. Leave the route through the work open, then test what comes back.
Build your briefI built an AI team around the way I work
What earned a place, what I got wrong and why the order of the work mattered more than the number of agents.
See what earned a placeWhat does a CV become when you can talk to it?
ChatGPSteve tested whether a source-grounded conversation could replace the usual hunt through roles, dates and project descriptions.
See the ChatGPSteve experimentWhy your first AI project should save hours
Choose work small enough to test, important enough to matter and measurable enough to earn the next experiment.
Read the eight-week approachWhy CRO people are the best AI adopters
The advantage does not come from knowing more tools. It comes from knowing how to test a claim before turning it into a workflow.
See the experimentation habitsAI is changing buyer research before the first click
Buyers increasingly form a view before reaching a website. This turns the change into an experiment you can run rather than a trend you have to believe.
Run the buyer-research testA test gives you one result. A system gives you that result every week. These two paths take you from the first to the second, on your work rather than mine.
Five steps. One proven gain at a time.
Pick a real task. Test an AI approach. Measure the difference. Keep what works. Stack the win into the way you work, so the gain outlives the experiment.
Brief the result. Leave the route open.
Turn the outcome, reader, source, constraints and checks into a brief you can reuse every time the task comes round. The builder runs in your browser.
Get the next experiment, not another list of tools.
Every fortnight I send one real AI experiment: the task, the test, the number and the honest verdict.
You also get one experiment you can run on your own work.
Free. Fortnightly. Leave whenever you like.
The blog is bigger than the hub.
This page is curated. The blog keeps the complete record, including earlier writing on experimentation, product, UX, AI consulting and GEO.
Browse the full writing archiveBring the same discipline into your business.
I help firms turn the AI they have already bought into systems that pay back. We find the biggest opportunity, prove the value and build from there.
Explore AI consulting