Measuring the Right Metrics with AJ Davis: A Conversation on User Experience and Conversion

by | Podcast

Podcasts

Episode 33 of Rebooting Business with David Somerfleck, originally released as “Measuring the Right Metrics with AJ Davis”

Every storefront that ever stood and lasted for more than a few months was built to be seen. It was built to be entered, to be understood in the softly- silent single second between a stranger’s shoe and a stranger’s solitary decision, and for ten thousand years the architecture of commerce answered to the architecture of the eye. Then a second kind of storefront was born, one with no awning and no window and no warm light spilling out into evening streets, one assembled entirely from plucked pixels and patience and the quiet arithmetic of clicks. Most owners built theirs the way a man builds a mailbox, which is to say once, and then walked away whistling, certain the doors alone would do the selling.

They don’t, of course; and that’s the larger point. That’s the storefront AJ Davis has spent a career walking through with a clipboard and a cold, calculating eye. On episode thirty three of Rebooting Business, host David Somerfleck sat down with Davis, a conversion rate optimization specialist and the founder of Experiment Zone, and the two of them spent half an hour taking apart a question most businesses have never learned to ask out loud, which is why so many visitors arrive at a website, wander, and leave without ever laying a dollar down.

Davis came to that calm conversation carrying credentials cut precisely for it. She had led optimization strategy for Fortune 500 companies during her tenure at Clearhead, working with names as familiar as CVS, Steve Madden, and lululemon, and she had built the user research practice and the research team there from bare ground. Before that she served as the lead UX researcher on Google’s Optimize product, present from the first sprint week through the launch. She holds a degree from Wellesley College and a Master’s in Human Factors and Information Design from Bentley University, which is to say she was trained specifically in the science of friction, the invisible catches and snags between a person and a purchase.

What User Experience Actually Means

Somerfleck opened where every honest interview should open, which is with a demand for definition. User experience is a phrase that gets passed around conference rooms like a collection plate, and it is remarkable how rarely anyone stops to say what is actually in it. Davis answered plainly, without the padding of jargon, and she answered in pictures.

“User experience can encompass a lot of things, but it’s definitely navigation, it’s figuring out what is the site about, so it’s your perceptions of things, your ability to quickly understand, and then the little interactions as well. So what does it look like to figure out how to go from the home page to the page that you’re trying to achieve the thing on.”

Somerfleck pushed the picture further, holding a bewildering government portal up beside a cluttered retail giant, and Davis agreed that the whole span of the visit falls under the same wide umbrella. First impression and final checkout belong to the same discipline. Conversion, she explained, is simply a ratio and nothing more mysterious than that, the count of people who could act set against the count of people who actually do, whether the act in question is a purchase, a phone call, or an email address left behind like a footprint in wet sand.

The Levers Beneath the Revenue

From definition the conversation dropped quickly into mechanics, and Davis laid out the levers the way a machinist lays out tools. More visitors converting at the same rate. The same visitors converting more often. The same customers carrying more out the door per order. Three levers, no magic, and every business is pulling on at least one of them whether it knows the lever’s name or not. What struck Somerfleck was her refusal to bury clients in numbers, and her insistence on surfacing only the metrics that are burning.

“We will first do an analytics audit and we’ll take a look at a bunch of metrics that we know are important, but we’ll only surface the ones that we see are really on fire or the big opportunity or signaling there’s a problem.”

It is a small phrase and a sharp one. Most business owners are not starving for data. They are drowning in it, standing waist deep in dashboards they never asked for, and what they need is not another number but a hand pointing at the one number that matters this month.

Two Identical Storefronts and One Changed Variable

When Somerfleck asked her to strip the mystery from A/B testing, a term half his listeners knew only as a sound, Davis reached again for the storefront and built something small and perfect out of it. The metaphor does more work in four sentences than most textbooks manage in four chapters.

“What’s really neat about the internet is that you can basically create two equal looking stores next to each other with the same foot traffic nearby, and people aren’t aware of the two stores, and you change one variable. A/B testing is a mechanism that we can use very easily in the online space to make one small change and then be able to quantify the difference in what customers ultimately do because of that change.”

Two shops, side by side, identical down to the doorknobs. Change the banner in one of them. Count the difference. That is the entire science, and everything else in the discipline is a footnote appended to that one clean sentence.

The Three Things a Homepage Must Say Before a Visitor Scrolls

Pressed for something a business owner could act on today, without buying a tool or hiring a firm, Davis pointed to the strip of screen a visitor sees before lifting a finger. Above the fold, that old newspaper phrase carried forward into glass, is where a website either earns its stranger or loses him. A visitor should learn within seconds what problem is being solved here, in what she called plain English, and should be handed somewhere to go next.

“That section above the fold is so powerful, and what I mean by above the fold is visitors can see it without doing anything, without scrolling, without clicking. The first thing they see should include what this business is solving, what this business is, in really plain English, and then a call to action.”

Landing Pages, Websites, and the Confusion Between Them

Somerfleck raised a knot he had watched tangle a hundred networking conversations, the question of when a business truly needs a dedicated landing page and when it is simply buying a second front door for a house with only one hallway. Davis untied it by talking about audience rather than architecture.

“A lot of the times we want to be very targeted in our communication, and we want to give people just the core information they need so then they can take action and convert. And so a landing page can structure all of that on one page with the simplest, most straightforward information.”

The two of them worked the idea through a hypothetical, a page built to sell a digital marketing course to lawyers and to lawyers only, and arrived at a simple rule of thumb. The narrower the audience, the more a dedicated page earns the right to exist. A website speaks to everyone who wanders in. A landing page speaks to one kind of person, by name, and says come this way.

Touch Points and the Quiet Machinery of Trust

They turned then to touch points, that accumulation of small contacts a customer collects on the way from stranger to buyer, from the confirmation that arrives after a subscription to the tone of the note that follows a week later. Every one of them is a chance to be gracious or a chance to be forgotten. Davis was candid that the granular craft of email optimization sat outside her own specialty, and she declined to pretend otherwise, but she was clear about the weight email carries in the whole equation.

“Email is an essential piece of the business, and there’s so much variety in terms of the impact. The way we look at email is we’re asking how do people who come from email convert, and how do they behave differently than somebody who might have just googled you and found you.”

What the Pandemic Changed and What It Revealed

The episode was recorded while the world was still holding its breath, and the conversation could not avoid the great shove online that came with it. Davis described the shift with the fascination of a scientist and the restraint of someone unwilling to call a catastrophe a windfall.

“We have a tremendous shift in customer behavior which, as a human factor scientist, I find fascinating to look at. We are seeing huge shifts in what people are buying, what people are running out of. Consumers are looking for things that people are shifting to online, so we’re seeing a lot more traffic to e-commerce stores.”

Somerfleck asked her directly whether every business ought to sell online, expecting perhaps the easy yes, and Davis gave him the harder and more honest answer instead, agreeing in principle while refusing to pretend the rule was universal.

“What’s brilliant about digital is that we have a really clear sense of what’s working and what’s not. I would say the answer is yes with like a little asterisk that there’s probably some audiences where that’s not going to be helpful for them and would not have a good return on investment.”

She closed on advice aimed not at dashboards but at habits, and it is the piece of the conversation that has aged the best. Talk to your customers, she said, far more often than you did before, because the world stopped holding still and their needs went with it. Every dashboard describes yesterday. Only a conversation describes tomorrow.

What Has Changed Since This Conversation

Davis is a Wellesley College alumna, not, as the original recording suggested through an audio mishearing, a graduate of any institution called Will College. Her master’s degree in Human Factors and Information Design from Bentley University remains accurately stated in the original conversation.

Experiment Zone, founded by Davis in 2017 and already running at the time of this interview, has since narrowed its public facing shape. Rather than general conversion consulting alone, the company now produces the Experiment Zone Podcast, a series built around live teardown audits of direct to consumer ecommerce stores, alongside a separate program called Startup Insights aimed at founders in their earliest seasons. The free personalized report card offered near the close of the original conversation no longer appears in that same form on the company’s current site, and readers interested in a similar audit should confirm current offerings directly with Experiment Zone.

Related Posts:

 

 

Listen and Watch This Episode:

VIDEO

AUDIO

Sources Cited: