Illuminating Best Bets

In the movie An American Werewolf in London, two young Americans backpacking through the English countryside are warned by a man they meet in a local pub (colorfully named The Slaughtered Lamb) to “keep to the road, stay clear of the moors, and beware the full moon.”

Not long after setting off down the road, the two friends are so engaged in conversation that they veer off onto the moor (ragged grassland) on the night of a full moon and, of course, one of the two is soon attacked by a werewolf. In the movie, hilarity (albeit of somewhat ghoulish flavor) ensues. In launching products, the result of veering off course is far from amusing.

When you’re gunning for growth, it’s easy to find yourself in the moors, working madly on improving a metric that ultimately doesn’t matter. Picking the right North Star helps to reorient growth efforts to more optimal solutions, because it helps illuminate when the focus of your experiments isn’t producing the results you need.

Deciding to abandon a course of action or an experiment can be very difficult, especially if team members have become quite committed to a direction they’ve championed. When you’re committed to an idea, pressure and emotions can all too easily bias judgment.

As Facebook’s original growth team lead, Chamath Palihapitiya, wisely cautioned in one talk, “If you can’t be extremely clinical and extremely unemotionally detached from the thing that you’re building, you will make these massive mistakes and things won’t grow because you don’t understand what’s happened.”

To clarify how dedication to improving a North Star metric helps make difficult decisions about how to spend time and resources, let’s look at how the Airbnb founders decided to conduct an experiment they thought might generate more nights booked — their North Star.

Airbnb’s North Star for Growth

To begin, they looked at their data to identify markets where bookings were lagging and, to their surprise, discovered that New York City was underachieving. Clearly, New York is a major tourist destination, so they dug in, with early investor Paul Graham of Y Combinator, to analyze why bookings weren’t stronger.

Reviewing the apartment listings for the city, cofounder Joe Gebbia recalls that “the photos were really bad. People were using camera phones and taking Craigslist-quality pictures. Surprise! No one was booking because you couldn’t see what you were paying for.”

Graham recommended that the two experiment with a hack to improve bookings that was low tech and high effort — but it was fast to execute on and wound up incredibly effective. The cofounders and Graham wrapped up the meeting and booked a flight to New York.

Gebbia and his cofounder Brian Chesky rented a $5,000 camera and went door to door, taking professional pictures of as many listings in the city as possible. Then they compared the number of bookings for the listings with the improved photos versus the rest of the New York listings, and found that the new photos led to between two and three times more bookings, instantly doubling the revenue they were seeing from New York.

Scaling Toward the North Star

With their hypothesis that photo quality impacted bookings proven, they quickly extended higher-quality photography efforts to other major cities that were also underperforming: Paris, London, Vancouver, and Miami. The uptick remained consistent, and so to ensure the highest number of bookings Airbnb decided to create a photography program that allowed hosts to schedule a professional photographer to come and photograph their listing.

Launching in the summer of 2010 with 20 photographers, it grew to include more than 2,000 freelance photographers, taking photos of 13,000 listings on 6 continents by 2012. Was it cheap? No, but the improved listings led to a two-and-a-half-times- better booking rate that held up all around the world.

Gebbia and Chesky could have tried to ramp up bookings by instead focusing on greatly driving up the number of people searching for listings in New York. They might have sent more promotional emails featuring popular New York destinations, or paid for search ads to appear when people Googled destinations.

Maybe that would have increased traffic to the site somewhat, but the number of bookings would still have been limited by the lack of the appeal of the photos. By keeping their focus on improving the ultimate goal of increasing the number of nights booked, they not only improved the rate at which people booked, but also made any future increase in traffic they drove to the site more lucrative.

Facebook’s Secret to Avoiding Analysis Paralysis

Just as it’s easy to lose your North Star and find yourself veering onto the moors in pursuit of vanity or irrelevant growth metrics, it’s equally easy to get lost in the weeds of data analysis and lose the sense of urgency to start experimenting with ways to actually drive growth.

Alex Schultz notes that the team at Facebook’s early charge to improve one key metric, which Mark Zuckerberg decided was monthly active users, helped the growth team break out of “analysis paralysis.” Doing more and more data analysis can be an especially alluring trap; because delving into data is scientific, we can convince ourselves that we’re just being rigorous and don’t want to experiment without sufficient evidence of likely success.

One of Schultz’s favorite quotes when he talks about the growth process is of US World War II commander General George Patton, who said, “A good plan violently executed now is better than a perfect plan tomorrow.” The clarity of the North Star goal helps to keep data analysis tightly focused so you can get your high-impact experiment off the ground as quickly as possible.