Merchants Should Be Using The Scientific Method for Everything

Anyone with a modicum of exposure to the startup world has undoubtedly heard phrases like lean startup, agile, or fail fast. Hype and marketing aside, these are simply euphemisms for the scientific method, something all of surely learned in grade school.

As a fresh reminder, here’s what the scientific method is:

The scientific method is a process for experimentation that is used to explore observations and answer questions.

In practice, the scientific method helps us arrive at the correct – or most correct – solution for a given problem.

It starts with having a hypothesis. A hypothesis is a testable conjecture for why something is.

Then the observer runs experiments. The point of the experiment is to prove or disprove the hypothesis, hopefully as quickly and as cheaply as possible.

If the hypothesis is proven correct, then the hypothesis is considered less of a guess because there’s now observable data to back up the claim. If the hypothesis is proven incorrect, a new hypothesis is generated, and the process is run again.

I’m pretty big on illustrations to help make a point, so here’s a diagram for the scientific method

When we see merchants fail in aspects of their business it’s almost always because the scientific method was ignored. The decision-maker takes a position of “things were always done this way,” or “I know best” and objective data is discarded.

That’s totally bonkers considering brick and mortar businesses are such low margin operators. Everything should be optimized with the scientific method since the business is so fragile. Let’s run through an example to show how the scientific method can add immediate value.

The majority of merchants spend money on marketing. The recommended amount is somewhere around 5% of gross sales on an annual basis. How should merchants decide what to spend it on?

Well, if it were me I’d want to prove – as much as possible – that the marketing was working and calculate ROI. Otherwise it’s simply wasted money. But how do you know if marketing is working?

One way is to run a trial campaign with your loyalty program. The nice thing about owning a loyalty program is you have a sample set of highly engaged customers willing to serve as unbeknownst guinea pigs.

You must first have a goal in mind. Is the goal to increase revenue, profit, or something else? For measuring profit, you need to have an understanding of your cost of goods sold (COGS). This is much easier to calculate in retail than foodservice, where COGS calculations are much more time consuming.

Once you’ve decided on a promotion, you must let your loyalty members know. Hopefully these outreach initiatives are sent via a trackable medium like text or email. Thus you should be calculating the number of opens to build some metrics around the top of the funnel (I’ll have a complete graphic below).

Next, you need to see when loyalty members are converting. This will let you build an estimated sales cycle for your promotion: did members redeem your promotion in 24 hours, or did it take them an average of three weeks? Some of this can be influenced with the type of promotion you ran (i.e. did your promotion expire within a period of time to incentivize faster redemption?).

Once you’ve seen loyalty members converting, it’s time to determine if the promotion generated ROI. To do this, you must look at how much your loyalty members spent in redeeming the promotion, how much your promotion cost, and compare it to a control group: non-loyalty members.

To get away from theoreticals we’re going to invent a marketing campaign. Let’s say your promotion was sent to your loyalty members advertising 20% off anything purchased in the month of January. Our implicit hypothesis is that promotion will increase revenue for the business.

Over the month of January you want to track how much your loyalty members spent, and how much your average customer spent. Good loyalty programs will be integrated into the point of sale (POS) and will extract this information automatically. If not, you will need to spend considerable time in the POS reports to extract it. Keep in mind that you also want to be aware of needed adjustments. For instance, if you’re a restaurant and you’re running a promotion for dinner happy hour, you cannot look at guest checks for breakfast. Adjust accordingly.

Let’s say the average loyalty member spent $50 (check average) and the average customer spent $42. Once you discount that $50 by 20%, the loyalty member spent $40: $2 less than normal customers spent.

Further, you have a higher percentage COGS on that $50 purchase that makes the promotion less profitable. If we assume COGS is 50% for all purchases, that means there’s a COGS of $21 (50% of $42) and a COGS of $25 (50% of $50) for the purchases by a normal customer and loyalty member, respectively.

This means there’s a gross profit of 50% for the normal customer but only a gross profit of 37.5% for the loyalty member. I got those numbers by dividing the profit by the revenue: $21/$42 for normal customers and $15/$40 for loyalty members.

At this juncture we would conclude that the loyalty member promotion did not bring in a higher check average nor profit than a normal customer. Also, depending on the additional business costs – like labor and overhead – it’s possible that the promotions were vastly unprofitable, like running a daily deal (Groupon) or using demand generation platforms for online ordering (Grubhub).

But we’re not done with this example yet! The last measurement needs to determine if the promotion brought incremental business. That is, would your loyalty members have patroned had you not sent the promotion?

This one is a little tricker to pin down analytically-speaking. The most confident way to answer the question would involve looking at the shopping frequency of your loyalty members over periods of time when you were not running promotions and compare that to their frequency in January. The snafu comes in that sometimes loyalty members will not scan or use their loyalty cards when making purchases.

Once you do all the math – looking at the cost to run and support the promotion, the amount of revenue brought in and the amount of profit lost – you can decide if the promotion worked. Now you’ll know if that 5% spend on marketing is paying the dividends you expected, and if not you can cut it before it wreaks financial havoc on the business.

Lots of work, right?

Well the easier and more robust method of promotional analysis involves machine learning of POS data. I say it’s more robust because it can measure across a number of promotional efforts (not just loyalty members) and statistically eliminate noise. I say it’s easier because machines can do all the work and save you the hassle of rummaging through the data yourself.

How? You need a decent amount of historical POS data for comparative reasons – say 12 months. For businesses that have not been around that long there are ways to augment data from similar types of establishments but our research shows that confidence takes a nosedive.

Assuming you have the requisite amount of data, you can now ask the machine your hypothesis. Let’s say a merchant ran a radio advertisement. The merchant would then only need to specify the dates, let the machine know it ran an event, and the machine would analyze check averages and check volume trends, backing out influencers like weather, events and other externalities that could impact the data. If your POS system is storing costs, it will rummage through your COGS as well.

Machine learning can make the using the scientific method faster, cheaper and more pleasant for merchants. There’s less interpretation needed, and the ROI becomes much more transparent. With prescriptive analytics, the machine can even analyze promotions that have worked in the past, and recommend when to run them in the future, noting expected benefit those promotions will bring.

Merchants 10 years from now will be using these tools on a daily basis as POS systems will be expected to offer this type of data portability. Rapidly applying the scientific method to fine tune many parts of the business will help keep merchants more competitive, solvent and growing.

How many of today’s merchants will be able to trust data science is anyone’s guess. But at a minimum they should refer to the scientific method for making all their decisions.