You are climbing up the steady incline of The Signal Ridge Trail when you are met by a hiker doubling back to a fork in the route which has been ambiguously signed. The trail notes you pull from your pocket indicate that you are to bear left at a trail split .8 miles from the trailhead.
“I think it’s this way,” he says, his voice pressured and his pace the pace of someone who has made a mistake and is trying to undo it as quickly as possible. He’s got worn-in boots and a decent pack. His legs look like hiker’s legs. The calf muscles are knotty.
He rounds the fork, bearing left rather than right and, indeed, this direction appears to take you up towards the ridge and then the summit. You can see the shoulders of the mountain through the canopy.
Turning towards your hiking partner, you both wordlessly decide to follow him. The trails are equally worn and inviting. The one he has chosen follows the small river, usually a favorite tactic of trail makers. All very two-roads-diverged-in-a-yellow-wood.
You step over a cluster of birch branches on the ground and follow the trail. The hiker disappears around the bend. Twenty-five feet later, something just feels “off.”
You’ve come around the bend and the wide, well-trodden path suddenly creeps in. It’s not much at first, just that the tall grasses and wildflowers lean into path before you. Soon the smooth trail begins to look more like a dry riverbed, which isn’t a problem . . . but what is a problem is that the grasses growing up between the stones are fresh, unscathed tufts.
The hiker, having crossed the dry bed, disappears into a hole in the trees where the trail continues. But even that looks “off.” New foliage drapes languidly over the trail.
You stop. Perched on two rounded stones. The breeze lifts the leaves and bends the blades of grass. You look at your hiking partner. Do you go on or do you turn back?
One of the main tenants of outdoor survivorship is that it is essential to observe your environment. Closely. But observation is only half of the required skill. The other half is an unbiased interpretation of the data you are being given.
Wilderness mishaps and death stories are filled with cautionary tales of people who blindly ignored obvious signs of risk and danger. And, when I say blindly, I mean they made the types of decisions that cause people – those safely listening in their armchairs and far from the buggy, evening woods – to gasp, agog, and say, “What were they thinking?”
How could they have ignored the clouds building in from the north and their thick, deep gray, color? Or the speed and size of tree limbs rushing along in a swollen river? Or the frayed webbing of their climbing harness? How could they have not understood the effects an unseasonably warm and sunny March day would have on the snow pack? And, on a recent SAR rescue, how could a woman have removed her foot traction and chosen to run across a flat stretch of ice and not seen the standing water that would turn that innocuous ice patch into a leg breaker?
The nearly-obsessive pursuit of a Goal, which is a hugely powerful brain process (and has, from an evolutionary standpoint, been key to our success as a species) can become our worst enemy. On the larger scale, this obsessive goal-seeking neurological process ensures that we don’t stop until we’ve secured food, shelter, water and a mate. But that same mechanism can cause us to ignore information that doesn’t fit the Plan or the Goal. It causes us to take square pegs and jam them into round holes until they appear to fit.
Another anecdote: I once found myself lost. I’d come to a river and, looking at my map, it appeared that I was close to where I wanted to be. However for that to be true, I should be on the other side of the river. The river should have been on my right the whole time. I stood there with the river bubbling pleasantly on my left.
“Oh, I must have crossed the river at some point . . . when it was smaller.”
“Right, that’s what I did. That little thingy that I stepped over, that was the river.”
“Okay, cool. Yeah, I know where I am.”
“I can cross back over this and I should be able to cut through the woods and be back where I was supposed to be in a few hundred yards.”
“Yeah, cool. Yeah, I know where I am.”
But I didn’t know where I was. I wasn’t even close. Looking back on how my brain reasoned this wrongness is a bit scary. It completely lied to me. I was warping the data I had into whatever I wanted it be. My brain wasn’t looking at the territory anymore but the map I’d constructed in my head that I told myself was the territory. The false map that justified my deep desire to make what I saw what I wanted to see. Had I followed that map, I’d have been truly lost.
In the end, I reluctantly decided that I wasn’t sure enough to go further. Barely decided. I turned slowly and retraced my steps through the woods back to where I’d departed from the known and right. It bagged my trip and I was supremely frustrated. The river I wanted needed to be on my right and no, I hadn’t crossed it. That little stream had not been the river. And, the river I wanted, according to my map, was larger than the one I’d been looking at. By a lot.
Selection bias is the tendency to validate and choose only those things that align with our desired map of the world. In software development, and in particular, when working within The Lean Startup methodology, selection bias means that you shut your eyes and ears to valid feedback that doesn’t match what you expected you’d hear, believe you need or that negates your “vision.”
Because collecting feedback on an iteration of an application and deciding, solely from the measurement of that data, if the experiment worked is a cornerstone of the process, Selection Bias effectively kills The Lean Startup methodology. It renders it impotent. You can’t pivot and you can’t know if you are “on-track” or “off-track” because you’ve already predetermined what the track is. You’re going to go crashing through the woods only seeing the “trail” and not an “abandoned trail you should not be on, goober!”
To effectively use The Lean Startup methodology, you must view and understand all data before you without bias.
Sometimes that’s not comfortable. Sometimes, it means you have to retrace your steps and bag your weekend. But no one wants to be lost in the woods and no one wants to die in the woods because they chose their vision over reality. No one wants to hear from their users, “But that doesn’t make any sense . . . what were you thinking?”
My hiking partner and I talked briefly. I laid out to her why I felt this was the wrong route. She was less experienced than I. We could hear the other hiker moving his way up his trail. We doubled back.
On the way back to the trail split, I noticed the cluster of birch branches pulled across the path. A classic, but subtle, trail marker. We bore right and I said I thought the trail split we wanted was farther up. We surely had not come .8 miles. We crossed a very NOT dry riverbed. It was not easy and it made sense that the other hiker had retreated. The trail on the other side of the river wasn’t visible at first but as we neared the shore, a blaze peaked out at us, marking the path forward.
This was the right way. The anxiety in my shoulders settled. It feels good when you know you have obeyed your observations. I have yet to have that practice go unrewarded.
We worried about that other hiker for a few hours, hoping to see him come up behind us. About a mile from the summit, I decided that if we didn’t see him before we reach the bottom and if his car was still in the lot, I was going to call 911.
On our descent, after a nice lunch and some great views, we came upon the hiker sitting on a rock about half a mile from the summit. His shins were scratched and bleeding. He was disheveled and winded. I didn’t have the heart to tell him he had a dead leaf stuck in his beard.
“It was not that way,” he said as he breathed heavily into his water bottle.
You can read Parts I and II of this series here: Kill Your Darlings: Exploring “The Lean Startup” through Nature, Part I; Four Pounds of Nuts: Exploring The Lean Startup Though Nature, Part II
