A/B testing in real life, and Real Life in A/B testing

You knew I had to show you the pictures eventually.

It would be prejudicial to comment before showing you the two photos.

My new-ish and very bright friend Ann Chao (HBS ’13; CEO of the company behind Cadenza, the award winning musicians’ practice app) recently had professional photos taken and informally polled her friends to inform her choice between the two best shots. Many more responses later than expected, this became a learning moment:

Following my previous post on “doing things we love within the things we love,” this has been my creative rumination for this week. It was fun to study people’s perceptions across various lines. And to note the tension between gendered descriptors like “warm and inviting” and the supposedly gender-neutral but usually-male position of a CEO or startup founder.

Check out Ann’s whole post for some of the qualitative and quantitative artifacts that emerged along different lines of her social graph. I will reproduce here the excerpt that snapped me from bemusement back into work mode.

I wondered if people who I met at different phases of my life would have different opinions. The number of respondents among my high school, college and HBS circles were roughly the same.

  • My high school friends had a strong preference for #2 (80%).
  • My college friends were almost evenly split between #1 and #2, leaning slightly toward #1.
  • My HBS friends also had a strong preference for #2 (77%).

Not sure how to explain the high school and college difference. But for business school, I presume that most of my friends have a pretty solid idea of what a professional headshot should look like, and #2 fits the bill better.

Suddenly I both actually cared about someone else’s headshots, and found myself in the throes of my own learning moment, to boot.

Ann, I can advance some theories behind the break in opinion between these two photos.

  1. Let’s all please take a moment to note that none of this would matter if Ann’s technical, interpersonal, and ethical values were not top-notch. They are. Little moments like this, where we see the richness of thought behind a fleeting decision or “minor” detail, are deeply telling moments about the measure of a person. (Ann’s more introspective version, “doing things we love within the things we love,” is admittedly more catchy.)
  2. College is the great experimental space of emerging adulthood, whereas high school and business school (I have also heard this about law school, from my brother and others) can be a bit more concerned with a certain kind of branding relative to the social context. Those who remember Ann from each of those periods likely made their selection based on the values they shared at that time.
  3. I may as well just step in it. For the record, these are things that have literally nothing to do with Ann’s skills, but we are already talking about “looking good” as opposed to, you know, anything else.
    • Ann is Chinese-American.
    • Men, “artsy” types (her word, and I stand by it), and more strongly tied friends all showed effect sizes for preference of the “warmer” #1 photo, with the common observation that it has an authentic or “eye” smile.
    • It is very astute of Ann to consider the gender implications of her visual identity as a startup CEO. For whatever it’s worth, I personally chose #1 because of its warmth, taking the professional context as obvious from her attire.
    • However: the eye smile causes a visual narrowing of the eye to the outside observer. Some visual stereotypes still propagate that carry often quite ugly messages; Ann, as previously stated is Chinese-American. Ann having many Asian-American friends, it is not impossible that this cohort featured a prioritization of visual “eye volume” or smize over warmth or affinity.
  4. I must contend in addition that, if we are already in the business of optimizing our physical appearance (and body language, and speech patterns) for branding against other socially normative values, “racial” physical features, along with all the other sensitive indicators, are fair game.
    • I play this game too. With my attire, or my voice, or my writing, or my height, or my business card, or my stated hometown–or the questions I ask you about yours.
    • You play, too. So even do those individuals who boast “ideal” features.
    • None of us can “change society” by ourselves, but we can change our product space, and the lives of everyone around us, for the better.
    • Bias may never go away, but injustice or distaste aren’t the point of our optimizationsSuccess is.

This afternoon’s chance reading has crystallized an idea for me. I believe that the time for questioning the rules of the game comes less often than we think, and oftentimes not in the way we expect. That’s because the game started long before we were born, and it never ends. The most effective question we can ask ourselves is that of what kind of players we intend to be.

In this game, momentum shifts often. Every play is a scoring play, for someone. We adjust our gameplan–our strategy and our brand–on the fly. Everything we say and do in our careers, and indeed in our interpersonal lives as a whole, has informational value that translates through our own reckoning, however oblique, into the real value we seek.

Everything is communications.

And all communications, then, have underlying analytics.

The velocity of technology and information has created an era in which we can all publicly, commercially be as personally astute as Lincoln, Oppenheimer, or Jobs. Or we can be exposed as tone-deaf on the scale of Andrew Mellon, Robert MacNamara, or Steve Ballmer. Consider that each of these historical examples, both the successful and unsuccessful, had access to the best data available of their day.

Then and now, data requires discernment.

Then and now, we only find true understanding of our work if we know where, when, and why we are searching for it. Today, however, the entirety of the decision space has collapsed to within arm’s reach for those who will grab it–even as the gravity of this transformation now demands of us an extraordinary density of thought. What was true all along is now clear: as areas of expertise based on optimization, communications (or sales, marketing, business development, product management, social media, content strategy, UX, design, doctrine, plans, legislation) and analytics (or KPIs, data-driven strategy, data mining, data science, business intelligence, usability research, behavioral economics, sentiment analysis, operations research) are two sides of the same coin.

Communications and analytics are the same thing.

Real life doesn’t come to us in neatly defined campaign cycles (unless you’re in politics) or environments amenable to conventional statistics, although we have acted as though it does. The real questions of leadership, strategy, and influence haven’t changed per se, they’ve simply helped bring each other to the forefront.

How much–and how granularly–can you capture?

How fast can you optimize back into new value-creating actions?

And of course:

Can you ask the right questions?

How hard, fast, and deep can you think? Apologies to Barry Goldwater, but extremism in the pursuit of value is no vice. And moderation in the pursuit of vision is no virtue.

Ann, your piece obviously got me to thinking. And for that, I thank you.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s