ICE scoring tactics

ICE Scoring Tactics Would Get You Killed On The Battlefield

Ask any self-proclaimed ‘growth hacker’ or ‘data-driven’ marketer how to achieve your marketing goals, and chances are their growth strategy will involve ICE scoring marketing tactics.

ICE (Impact, Confidence, and Ease of implementation) is a framework for taking all the ideas your team has for growing your company, and prioritizing them using data.

  • Impact: if this works, how big is the impact?
  • Confidence: how likely is this to succeed?
  • Ease of implementation: how much resource will this take?

A high number is always good, and a low number is always bad. You average the three scores and rank your tactics highest to lowest to prioritize what to do next.

This gives you a big list of tactics, prioritized with the ones that’ll have the biggest expected impact for the least cost at the top.

Seems reasonable; what business doesn’t want to do more with less?

BUT decision-making pros don’t use anything like this model.

That is, according to Gary Klein, who studied how firefighters, critical care nurses, pilots, nuclear power plant operators, battle planners, and chess masters actually made decisions.

When asked, many experts couldn’t remember making decisions or even considering alternatives. Instead, they visualize potential solutions in their imagination, and choose the ‘first workable option’, acting upon it immediately. They generated alternative plans only when their first actions failed.

Yet these experts routinely make good decisions.

They often must juggle complex goals in high-stakes, time-pressured situations that are shifting by the second, where a bad choice can mean loss of life. Their know-how allowed them to read a situation quickly – even immediately – to identify patterns and act.

They don’t let themselves get paralyzed by self-doubt or waste any time finding the best possible outcome.

“A good plan violently executed now is better than a perfect plan executed next week.” – General George S. Patton

On the battlefield, only a sniper has time to calculate the complex physics of the arc of their bullet based on distance, wind-resistance and the curvature of the earth.

The rest of the soldiers have to point, shoot and pray.

That’s probably why it took over 250,000 rounds of ammunition per enemy combatant killed in Iraq and Afghanistan. If you think that’s wasteful, consider another quote from Patton:

“…remember that it is much better to waste ammunition than lives. It takes at least eighteen years to produce a soldier, and only a few months to produce ammunition.”

If you can waste a few bullets in the enemy’s general direction and that gets them to take cover while you get your men to safety, well, that’s a good use of bullets.

Can you imagine how ridiculous ICE scoring tactics would be in a war?

The general sits quietly in his study, assigning an Impact, Confidence and Ease of implementation score to each tactic while his men stare at him in amazement.

  • Air Strike: 9 | 10 | 2 = 7.0
  • Mortars: 4 | 5 | 9 = 6.0
  • Tanks: 6 | 4 | 5 = 5.0
  • Troops: 3 | 6 | 3 = 4.0

“Ok, great, air strike it is! Call it in boys!”

The men shuffle nervously as the general surveys the battlefield again, sees the effect of the air strike, and recalculates his tactic ICE rankings.

“Hmmm, now the enemy is hiding in their foxholes, Air Strike and Mortars lose -4 Confidence and Troops gets a +7 in Impact. Time to send in the Troops!”.

I bet a general like this wouldn’t last very long.

As Sun Tzu says in Art of War, “Tactics without strategy is the noise before defeat.”

But what does that mean?

What is the difference between tactics and strategy?

Wikipedia.org differentiates them as follows:

“…tactics are the actual means used to gain an objective, while strategy is the overall campaign plan, which may involve complex operational patterns, activity, and decision-making that govern tactical execution.”

It’s painfully obvious that simply ranking tactics can’t be called a strategy. It’s far too simplistic; it doesn’t take into account how tactics are interconnected towards achieving the goal.

This is the type of ‘strategy’ a child might use; making formulaic decisions based off game-like mechanics that don’t exist in real-world situations.

I’m no wartime general, but a real strategy would sound something more like this:

“The enemy line looks weak at this bridge; the reinforcements they were expecting got held up by our air raids last night and aren’t expected until tomorrow. I say we amass troops higher up the river where the mud is thickest and any potential counter attack would be hindered. We lay four rounds of mortars (saving plenty in reserve) on their lines opposite. Right when the mortars finish and they’re expecting us to advance, we send in an air strike to the side to take out the bridge. Our tanks can be outfitted to get through the water at that depth and do our dirty work on the other side with minimal loss of life. They’ll never see it coming and they have no good option for a counterattack without getting bogged down in the mud and risking heavy casualties.”

The general couldn’t possibly calculate the all probabilities of success. Many things are unknown or could still go wrong. Neither side ever has complete information, and even modern warfare’s technological advances still haven’t lifted the fog of war.

The plan takes into account the terrain, weather, ammunition, weaponry, logistics… too many variables to count. It anticipates the enemy’s actions and reactions, using imperfect information and the limited resources at his disposal to come up with a creative solution to achieve victory.

The plan is complex yet clearly communicated in a way even the best machine learning algorithms couldn’t reproduce. It is uniquely human in a way that’s hard to define.

Most importantly, it’s less likely to get him and his men killed.

Why Is ICE Score So Prevalent?

Just like ‘Growth Hacking’ was invented to get more meetings with marketing-phobic tech founders, ICE scoring is a marketers’ solution to getting engineers to listen to their requests.

Here’s how the process works:

  1. Brainstorm a big list of ideas to be tested (Backlog)
  2. Estimate an ICE score (out of 10) for each test idea
  3. Prioritize test ideas by ICE Score and business focus

So you’re looking for a high Impact, high Confidence or chances of success, and high Ease of implementation – meaning a low cost and/or not time-consuming to get done.

Ranking tactics by these factors gives you a sense of which tactics will give you the most impact for the least cost and effort. What are the quick wins? The bigger long-term strategic bets?

It’s a great system.

It has instant appeal to those who want to appear ‘data-driven’ with their decision-making, which can have a powerful effect.

Rather than deciding via the HiPPO method (Highest Paid Person’s Opinion), ICE ranking lets the best ideas rise to the top, no matter where they came from.

You can control a meeting or even the entire roadmap just by being the most organized in the room; most higher-ups won’t or can’t take the time to push back in individual weights.

It means nobody feels like their ideas aren’t heard, and you always have a ready-made list of what’s the most important thing to work on next. They had their chance to contribute, and they can’t have their feelings hurt if their ideas didn’t ‘objectively’ get a high enough rank.

At the very least if you make the wrong decision with ICE ranking, you have something to show your boss and prove your diligence in order to cover your ass.

For agile teams it’s a godsend because so long as they’re always pulling a set number of tasks from the top of the list, they know they’re working on the most valuable tasks. They can comfortably push the rest of the ideas to next week, next month, or never.

It’s no wonder why it’s so popular with startups. Hubspot, Drift, WP Curve (and countless others have written about how they use it to grow).

It’s so popular in fact, that Sean Ellis (who coined the term “Growth Hacking”) created Growth Hackers Projects, software designed specifically to help companies manage this process.

prioritize by ice score seanellis
image via GrowthHackers

It’s no accident that ICE ranking and the Growth Hacking movement came around when Silicon Valley engineers at Dropbox, AirBnB, and Facebook got put in charge of marketing.

ICE scoring is an engineer’s way of approaching strategy.

Break a messy real-world problem into smaller parts, assign values to each part and run the calculation to predict the potential outcome; boiling life down to a Fermi problem.

I’m not saying this approach doesn’t have value; after all, you want the Engineer building a bridge to adhere to the laws of physics to make sure it doesn’t fall down.

The technology companies these Engineers have built are amongst the most valuable companies in history.

top 20 internet leaders
image via Visualizing The World’s 20 Largest Tech Giants

Of course, this isn’t unique to marketing – in other industries, you might know this as a Cost-Benefit Analysis, a fairly standard practice.

Why Optimization Is The Wrong Approach

In economics, they call this approach to decision-making ‘rational choice theory’. In that model, you identify and evaluate your options, weigh different aspects, produce ratings and choose the alternative that scores the most points.

What the experts Gary Klein studied were doing, is called ‘satisficing’ and it’s a dominant plan in conditions of uncertainty, versus formal analysis, or ‘optimization’, which only works in simpler models of reality. For example, rational choice theory assumes humans are rational, that everyone has perfect information and that it costs nothing to gather that information.

Experts know that nothing comes for free; gathering accurate information has a cost.

Whether it’s an emergency situation where every second counts, a long drawn out meeting or weeks spent in annual planning, you should always be calculating whether it makes sense to keep searching for information, or just make a ‘good enough’ decision with what you have.

I’ve argued this same issue applies to marketing experiments in ‘cascading significance’.
i.e. How do you choose when to end a test as conclusive, when it costs time and money to keep it running?

Experts are good at making that tradeoff. Beginners get struck with analysis paralysis.

Daniel Kahneman, in Thinking Fast and Slow, labels this System 1 and System 2 type thinking. System 1 “is the brain’s fast, automatic, intuitive approach” versus System 2, which is “the mind’s slower, analytical mode, where reason dominates”.

It’s System 2 that gets derided as being emotional, irrational and fallible. It’s the reason we eat so much sugar, don’t save enough for retirement or get tricked by fake news. There are a whole host of cognitive biases behavioral scientists have discovered; irrationality and quirks in the way System 2 works. Engaging the cold, hard logic of System 1 is seen as the only thing that saves us from our base, animalistic side.

Contrary to popular belief, System 2, the ‘irrational’ side of our brain, is actually where the bulk of decisions are made. In fact, there’s evidence of the tail wagging the dog: System 2 will justify decisions with analysis that were actually made by System 2 for irrational reasons, without conscious knowledge. System 2 covering up for System 1.

In a study of people who had the two sides of their brain severed (to prevent epileptic seizures), scientists were able to send a message to one side of the brain, such as “Go to the water fountain down the hall and get a drink.” without the other side seeing it.

They’d then ask the other side of the brain, that didn’t see the message about the fountain, “Where are you going?”. Instead of admitting they didn’t know, the subjects shamelessly fabricated a rational reason, like, “It’s cold in here. I’m going to get my jacket”!

System 2 is in control because it has far superior computing power; it’s keeping your whole body running and coordinated, and can react to split-second decisions before you’re even consciously aware of what happened.

In Snowden’s Four Ontologies, it’s only ‘Complicated’ problems that System 2 is good at solving; problems where the answer is knowable and analytical / systems thinking will be useful. System 1 is handling any ‘Simple’ problems automatically, as it can match the problem to the known solution. Chaos can’t be legislated for, so it’s really Complex problems where strategic thinking comes into play.

snowdens four ontologies
image via HowToSaveTheWorld

Complex problems are by definition unknowable, so System 2 can’t be in command. However it can help; we’re not looking for simple pattern matching, we actually have to think of a logical way to ‘probe’ and then maybe even consciously interpret what we’re sensing before responding. This is the mixture of art and science that leads to good strategy.

A note…

Some of you will recognize elements of the Lean Startup principles in this. Building a business is indeed a ‘Complex’ problem; it’s unknowable because customers are, as Jeff Bezos calls it, divinely discontent; their needs aren’t static, they update and change over time.

lean startup process kissmetrics
image via KISSmetrics

Just like the prescription for solving a Complex problem is to probe > sense > respond, Lean Startup advises you to build > measure > learn. They recommend launching early and launching often, running small experiments (probe) to measure what’s working or not (sense) before you learn what scales (respond), confident you’ve found a winning product for now.

Rationality Is Bad For Survival

In practice, the nuance of each assertion and assumption that goes into building a strategy, combines the best of Art (System 1) and Science (System 2).

The general knows fighting in mud leads to higher casualties. Moreover, he knows his enemy also knows that and can rely on this information to protect his men from retaliation after his plan goes into effect.

He might run a cost-benefit analysis or employ something approaching rational choice theory, but that would just be one input that could be overridden by emotion and subjective choices.

You might not think of soldiers as driven by emotion. However, if they weren’t driven by emotion, why would they risk their lives to save fallen comrades at enormous, and irrational, risk to themselves?

Why would the morale of troops have so much impact on outcomes? Why would a General leading the charge from the front have any effect at all? How could attrition warfare “[wearing] them down to such an extent that their will to fight collapses.” work if emotions weren’t a factor?

In 1976 the US launched Operation Paul Bunyan, where they sent in 110 troops, (64 of whom were Korean tae kwon do experts), 27 helicopters, 3 B-52 bombers to… cut down a tree.

The operation was a retaliation for North Korea seriously injuring 8 men and killing 1 in a previous attempt to cut back the leafy poplar tree in the demilitarized zone.

…and it worked.

It showcased American strength and power while mitigating the risk for escalation into war. According to an intelligence analyst monitoring the North Korea tactical radio net, the accumulation of force “blew their… minds”.

Why did that work?

Why is using emotion a dominant strategy?

This uniquely human ability to model and predict complex situations, anticipate and account for potential future scenarios, is key to our ability to survive and thrive as a species.

Say a general did find a way to simply calculate the objective cost and benefit of each tactic and then blindly implement the results. Wouldn’t his enemy would soon figure out his methods, and have a perfect map of his opponents’ expected behavior to use against him?

Rory Sutherland, vice-chairman of Ogilvy & Mather, has connected the dots between the irrationality of war, evolution and marketing for us.

“It’s impossible for anything rational, to successfully evolve, because the byproduct of being rational and efficient, optimally rational and efficient, would be that you’d be predictable. And if you’re completely predictable, you’d be dead.”

If you dig a little deeper it’s an understanding of psychological biases, not just engineering prowess, that power tech giants like Apple, Amazon, Google, and Facebook.

Quirks of human nature where our System 1 brain fails us are taken advantage of by smart techies, in the same way evolutionary rivals would exploit any advantage against us.

Being irrational might be good evolutionary strategy, but being ‘predictably irrational’ as Dan Ariely calls it, means we can be taken advantage of.

If Engineers think about marketing at all, they see it as a conduit for information about the product, which in their mind should speak for itself, without any of the fuzzy brand stuff.

But that doesn’t explain why the U.S. military sent hundreds of personnel and armaments to cut down a tree.

Rory Sutherland argues that efficiency is overrated, and that excess and superfluity are weapons that marketers surrender at their peril.

“Knowing that the seller has faith in their product is a hugely valuable piece of information,” he says. “In luxury goods, for instance, the ad says almost nothing; the cost of the ad almost everything.” Biologists regard the peacock’s tail as an expensive and so unfakeable signal of fitness – a sexual status symbol.

…A flower is basically a weed with an advertising budget.”

When the U.S. military deployed all of that resource it was essentially advertising its seriousness and prowess to the North Koreans in a way that was expensive and risky to fake.

Knowing that, and predicting the North Koreans would back down, took real strategy. That’s not something you can capture by ICE ranking tactics.

Even if the General could attempt ICE ranking, he’d find what most Growth Hackers do; his number weightings are basically subjective anyway.

He might stay honest at first, but in the heat of the moment would eventually probably be tempted to rework his initial weightings when his favored tactic doesn’t rise to the top. This is behavior I’ve observed in every Growth Hacking team I’ve worked with.

Not to mention he would rank each tactic differently to his fellow officers or the men that had to carry it out. Any one man’s rankings might change from day to day, minute to minute, second to second. Getting an accurate, objective read of rankings in time for a decision? Impossible.

Why would the rankings be so subjective and inconsistent? They’re delivered by System 1. This means the ICE ranking strategy is only giving the illusion of being ‘data-driven’ and powered by System 2. In fact, ICE is just capturing decisions already made emotionally by System 1.

How to go from Beginner to Expert

Beginners are always in System 2 mode because the situation is unfamiliar to them; it’s not a pattern they recognize as being similar to anything they’ve seen before.

So they resort to the slower process of listing positives and negatives, calculating expected values and taking the slower but more logically defensible path.

Have you noticed that experts rarely give much justification for their decisions unless forced to?

When they do give you notes on their thought process, its likely wrong. They might not even be consciously aware of it, but mostly the logic was constructed after the fact to support the decision they already made; System 2 covering up for System 1 again.

Klein’s key finding was that novices use optimization as their prefered strategy, and experts abandon optimization and go with their gut. This concurs with the Dreyfus model for progressing from novice to expert via a series of stages.

You start as a novice by following simple recipes (tactics) and progress to advanced beginner by recognizing the right recipe to switch to when you find a problem. Approaching competence through inventing a series of rules for when to switch recipes, you become proficient by gaining the ability to target the right goal. You can eventually call yourself an expert when, through enough deliberate practice, you can do all this on auto-pilot, intuition or muscle memory.

You get from one stage to the next through deliberate practice. You’re trying to build this pattern matching ability into your System 1 thinking, so take care what data you feed it. One technique I have found personally useful is ‘learning by osmosis’; maximizing your exposure to the thing you want to learn to intentionally build up those feelings of what is normal or not.

Not understanding this, is why beginners get so discouraged by learning a new skill.

Like Jeff Bezos illustrated in one of his annual letters to shareholders, most people think they could learn to do a perfect handstand in about two weeks. The reality is that it takes about six months of daily practice.

So a beginner sees an expert effortlessly doing a perfect handstand or shooting from the hip when it comes to pivoting marketing strategy and try to do the same, failing miserably. Or maybe they get disillusioned, like I did early in my career, by the expert’s inability to explain why they did something, and conclude it’s all just subjective and the emperor has no clothes.

What’s really years of deliberate practice culminating in an expert’s System 2 being able to accurately pattern-match and act instantly, can look indistinguishable from bullshit to a skeptical novice. The expert sees the outcome as a ‘no-brainer’ but for the novice, it’s not.

Experts are good at recognizing points of leverage where the upside more than compensates for the downside.

For chest compressions to work they have to be so aggressive it often breaks their ribs. When the upside is restarting their heart, a trained professional will take that tradeoff; a novice frequently fails to push hard enough.

Even in the investment world, experts don’t try to predict the market; they look for points of leverage or asymmetric risks. If they can spot an opportunity to make a lot of money if it goes well, but only lose a small amount of money if it fails, they’ll take that bet.

The Right Infrastructure For Strategic Effectiveness

You might think the military has tight control over their troops, and of course they do enforce strict discipline, but you’d be wrong if you thought all actions were dictated from the top.

Usually, Generals just focus on figuring out the strategy and make sure that everyone in their command knows it. Even with today’s technology it wouldn’t be possible, wise or prudent of them to micromanage soldiers on the ground under gunfire.

Instead, they practice ‘extreme ownership’ at every level, blaming operational failures on leadership for not communicating the strategy clearly, not the teams involved. They trust their teams to prioritize and execute tactics on the ground, so long as it furthers the overall mission.

Like Elon Musk, they understand that everyone in their organization is a vector; in physics terms that means everyone has a quantity of both magnitude and direction. Progress is determined by the sum of all these vectors. If some vectors are exerting energy in one direction, while others are doing so in a different direction, the organization won’t get from point A to B.

*For example, if an employee is pushing with a force of 10 in one direction, and another is pulling with a force of 10 in the opposite, the total impact is zero. To achieve maximum impact, all vectors must be exerting energy in the same direction, devoting their efforts to a common vision.

The tech giants adopt the same leadership approach as the military.

They set the strategy and get out of the way. As an example, Asana runs its company on buddhist principles, and the CEO Dustin Moskovitz couldn’t be more pleased than when his employees overrule his solutions.

Moskovitz’ prior employer Facebook’s strategy is to use technology to connect people, but don’t think Mark Zuckerberg is sitting there dictating or debating every change to the website.

“Instead of debating for days whether a new idea is possible or what the best way to build something is, hackers would rather just prototype something and see what works.” –Mark Zuckerberg

Giving employees full license to experiment as a publicly traded company sounds like a recipe for disaster, and Facebook has no doubt had its troubles. But even with the recent Cambridge Analytica scandal, the Russian ads influencing the election, do you know what hit the stock price? Lower than expected user growth.

The danger of not innovating fast enough is by far greater than the fallout from a rogue employee putting an experiment live you didn’t like. Like Boz, a VP at Facebook says;

“I know a lot of people don’t want to hear this. Most of us have the luxury of working in the warm glow of building products consumers love. But make no mistake, growth tactics are how we got here. If you joined the company because it is doing great work, that’s why we get to do that great work. We do have great products but we still wouldn’t be half our size without pushing the envelope on growth. Nothing makes Facebook as valuable as having your friends on it, and no product decisions have gotten as many friends on as the ones made in growth. Not photo tagging. Not news feed. Not messenger. Nothing.”

Another company whose stock price just can’t stop rising, Amazon, is run like a massive distributed machine. Employing principles of radical decentralization, hundreds of ‘two-pizza teams’ plug into the three main platforms; ecommerce, logistics and web services.

They launch thousands more experiments than Jeff Bezos could ever possibly be kept in the loop on, and share metrics with full transparency back to HQ. This is allowing the company to scale infinitely, with the profits from successful experiments reinvested into new ventures.

If you’re going to empower your employees to do permissionless innovation and execute on the tactics they think best accomplishes the strategy, you’re going to need to rethink your infrastructure.

In Engineer Steve Yegge’s colorful rant, he proclaimed that “Amazon does everything wrong, and Google does everything right”, yet Amazon does one thing that “makes up for ALL of their political, philosophical and technical screw-ups”: they built their whole company into a platform.

Jeff Bezos realized “he can’t always build the right thing”. By turning the whole company into a platform, don’t need to build one product and have it be right for everyone. They can just focus on enabling distributed teams and third-party developers to do it, and “it would happen automatically”.

Facebook built and open sourced PlanOut, an experiment framework that allows anyone to launch and report on an experiment without worrying about how it’d clash with the other thousands of experiments on the site.

How To Hire and Manage Experts

If you have a hand in designing how your organization works, chances are you’ll be hiring people with greater expertise than you have in some topics, including marketing.

It’s a recipe for frustration to not give those people autonomy over the area they’ve mastered. Recognize that as a non-expert in their area, you might be unqualified to judge their work! You might need to put hundreds or thousands of hours in to just understand what they’re working on.

The Feynman technique says that anyone who can’t express what they think, don’t really know it well enough. That may be true, but remember to be sensitive to the fact that you aren’t always working with a full expert in every area.

Your junior marketer might be able to get 80% of the way there (for 20% of the cost than an expert) following simple tactics they read from blogs. However, if you’re repeatedly asking them to justify their plans or build a wider strategy, they’re really going to struggle.

Remember it costs time and money to gather information, so you should treat every report and plan you ask for as a tax on the work they’re doing. In most organizations I know (and I’ve worked with over 200) this tax far outweighs the actual work they’re doing.

When you hire someone more senior, what you’re buying is someone more proficient at coming up with strategy and explaining it to you and the rest of the organization. Senior people in marketing very rarely do any actual campaign work; they spend close to 100% of their time planning ahead, checking actuals versus the plan, then adjusting as necessary.

On the other side; the bigger an organization becomes, the more the marketers have to be available to justify their strategy and tactics to non-expert people. If you don’t have this structure in place, you get non-experts overriding the work of the marketers actually doing the work, and performance suffers.

So the CMO is there to protect against the CEO, the Marketing Directors protect against the CMO, the Senior Marketers protect against the Marketing Directors and the Junior Marketers are free to get the work done.

It’s just as bad, maybe worse in agencies, because it’s easier to fire us if the client doesn’t like our plans. It’s not unusual for over 50% of the ‘work’ done for a client to be communication overhead. Weekly calls, monthly meetings, ad hoc requests, media plans, creative approval, scope creep; all of these things add up and ultimately mean less actual work for the same price.

Of course, this is all good intentioned; they just want to know the right things are happening at the right time to coordinate with the rest of the organization. Without documentation it can be hard to control quality or meter out discipline. Getting people to justify their approach up front makes sure we’re all on the same page, and helps us spot errors.

Just know that, from experience, it’s possible to more than double output by removing or relaxing some of these requirements.

For example…

With one of our clients, Booking.com, we tripled the number of marketing experiments running per month by cutting this unnecessary red tape. We’re now free to launch small experiments without any prior justification, and scale them up without a business case so long as they’re helping us hit our goal. They just check our creative for legal purposes (no judgement passed on if they like it or not), and follow our progress with interest as we find new insights into what’s working. If the 3rd biggest ecommerce company in the world can do it, you what’s your excuse?

How To Set Strategic Goals

If you’re going to take the plunge and empower your employees/agency to operate according to the principles of permissionless innovation, you’ve got to set a high-quality goal or objective.

Below is an example from 500 Startups of the OKR (Objective Key Results) framework that captures it nicely. You have a clear objective which would have a strategic impact on your business. There is a set time period, in which 1-3 months is usually enough to make it meaningful. You then set a range of key results (KRs) that would be a clear and measurable indicator of success, from just hard to hit, to very unlikely.

growth okrs 500startups
image via 500Startups

The objective is clear; the goal is to establish Facebook as a meaningful and viable acquisition channel. It doesn’t go into detail as to why this is an important part of overall company strategy, or why Facebook was chosen, but that’s enough for now.

But what does meaningful and viable mean?

The Key Results (KRs) are where you give life to the objective; a $22 cost per paying customer would be meaningful to the business. Even better if you can achieve a $15 CPA at 2,000 new customers per week.

Notice that the KRs are a mix of both efficiency and volume goals. Usually, in digital marketing at least, efficiency gets worse as volume increases, due to diminishing marginal returns. So by designing the KRs in this way you’re explicitly saying to focus on efficiency (cost per paying customer) first, then only on volume once you’re consistently below $15.

KR1 should be the minimum level needed to judge the objective a success; KR2 should be your realistic target performance and KR3 should be a massive win, worth sharing far and wide.

It’s important that the objective and key results be truly authentic.

There should be some existential or fundamental reason for setting the target in the first place. If you’re a startup and your investors tell you “you won’t raise your next round unless you get an acquisition channel working”, well that qualifies. But it can be something internal like “if we don’t get our churn rate down past 10% we don’t have a good enough product”. Finally, it can be a positive, audacious goal like “how could we hit 500,000 people using our software?”.

This is important because if the goal isn’t authentically tied into something that would be meaningful to the company, your smartest staff will quickly become disillusioned with it. Don’t treat goals like a negotiation; if you arbitrarily increase the numbers in a spreadsheet it won’t get people to work harder for long. If it has no basis in reality, they’ll either lose confidence in themselves or give up and blame someone else; you won’t get their best work. Good people don’t just stop when they hit the goal; if anything they keep going harder and really smash it.

You can stitch these objectives together into an overall growth story for your business. If you can relay the elevator pitch for each month and what you achieved, or plan to achieve, then everyone in the business can pull in the same direction.

If you don’t know what’s achievable, this is the perfect time to ask an expert for their benchmarks. Even if you have to pay them exorbitant sums of money, you’ll only need a few minutes of their time, and it’ll save you a lot of future time and money by setting the right goals. I make myself available to any strategist in my company to quickly ask me for a gut response to any plan, metric or strategy to see if it ‘feels’ wrong. It doesn’t take long and it save them hours or even weeks or months of hard work in some cases.

The trick is to find an expert that has absorbed the right type of information to prime their System 1 responses. Get them talking about their experiences, and make sure you capture any typical metrics they remember, and how long certain things took to work.

You’ll usually find their responses are refreshingly honest and they happily share with you some of the highlights and low points; this is gold.

When setting strategic goals, you should refrain from doing what most people do in this situation and say something like “ok you saw a $60 CPA, but my app is going to be so great that I think we can halve that…”. This is called the base rate fallacy and you should guard against it; go with the expert’s judgement unless there’s some major structural reason it’ll be different in your case.

How Growth Strategy Changes Over Time

Often people are unprepared for how quickly growth strategy can completely shift over time. When you make a 5-year plan, you’re really making a prediction of the future.

The further out you get, the less accurate the plan will get. In my experience, it’s usually ok to set monthly or quarterly goals, but any further out than that, the future is too hard to predict.

“Everyone has a plan until they get punched in the mouth” – Mike Tyson

Only 1 in 10 marketing experiments succeed, and every great campaign is built on the back of a lot of failed experiments. Given that they’re wrong more than they’re right, asking your marketer to predict performance years ahead of time can be a futile exercise.

Below is an example journey of a typical Ladder client I give to set new client’s expectations of what is likely to happen in an engagement with us. It gives them a good overview of how we might move from one objective to another as we learn more about what works or doesn’t.

  • Month 1: Client has <1,000 visits per month so we start testing with Facebook ads
  • Month 2: Facebook ROI is -60% due to low LTV, so we start testing Google Adwords
  • Month 3: Adwords is working well at 10% ROI, so we scale it and turn off Facebook
  • Month 4: Adwords scales, but ROI goes negative once we hit $10,000 spend per month
  • Month 5: With more than 10,000 visits per month we can now start testing conversion
  • Month 6: We start deploying SEO tactics to rank for the keywords that work in PPC
  • Month 7: First major win in conversion +10%, increases the amount we can spend on PPC
  • Month 8: We now discover a retention problem related to onboarding, deploy tests to fix
  • Month 9: Seeing success with SEO, still <5% of traffic but growing so we double investment
  • Month 10: With a solid base of existing customers, retargeting / CRM tactics become a focus
  • Month 11: CRM an instant success, LTV up 20%, making Facebook worthwhile to test again
  • Month 12: Facebook working, Traffic to 25,000/m, organic conversion and retention improving

You’ll notice we get a lot wrong; Facebook fails the first time, we don’t get a win on conversion for two months, and SEO is still only 5% of traffic three months in.

This is actually fantastic performance. If they’re not just showing off, most expert marketers would say taking a website from <1,000 visits a month to 25,000+ is a fantastic result.

Yet when I give clients this example, do you know what they say?

“Is there any way to speed it up?”.

Of course, I’ve seen some clients grow faster than this, but many more grow slower. In fact, the default is that you don’t grow at all; 75% of venture-backed startups fail. When your risk is death, you shouldn’t be spending any time worrying about 5-year forecasts; your goal should be intimately linked to your survival.

Because ultimately that’s all strategy is; an evolutionary response to survive.

So if you’re looking for a way to improve your company, you should focus less on planning ahead or tightly micromanaging what tactics are launched.

Instead, do what evolution does; keep producing and learn from what survives.