Calculating A/B Test Length

ab test calendarIf you were a fly on my wall, you would hear this:

Marketer: “Jared, could you get tell me how long I will need to run this test for?”
Jared: “Sure. May I inquire why you're asking?"
Marketer: “Because there are more tests I've put on the testing calendar right behind this one. I need to make sure this can finish up quickly.”

Is Testing On the Web Really “Free”?

free a/b test analystsShort answer:
Not unless you have magical oompa-loompas that you don’t need to pay, and these oompa-loompas are really good at persuasion architecture, design, analytics, & web development.

Reverse Engineering A/B Tests

Can you spot the difference?
Can you spot the differences?

One of the fun things I like to do when I have some free time is to try to learn from A/B tests other companies are running. My thinking is, "Why run a test when you don't have to?" There might be something you can learn and adapt to your business, or it might spark some new thinking.

Oktoberfest Video Interview: Jared Grabs a Virtual Drink with Matthew Wright (HP)

capple_0.jpgHave you ever gulped down an Apple Beer? Me neither. But Matthew Wright has. He also has some great experience in A/B testing at Hewlett-Packard. So pull up a stool and join our conversation. Jared Waxman chats with Matthew Wright, Manager of HP.com Worldwide Analytics, about A/B testing within large organizations.

As part of series on A/B Testing on http://www.benchmark-analytics.com these two practitioners share their thoughts on topics including: What are the benefits? What are the misconceptions? What to look for when hiring analysts?


Evaluating Test Ideas

A/B Test Ideas Are  Crap Shoot

In any organization that does a/b testing, ideas on what to test are rarely in short supply. Test ideas seem to fly out of every corner of the company. So if there are plenty of ideas floating around, what’s the problem?

Most of them are crap shoots. Blind guesses. Shots in the dark.

Take the Challenge: Are You An A/B Testing Ninja?

Choosing Your Testing Goals

Whenever I’m engaged by business leaders to help them with website optimization (a/b testing and the like), there’s usually an awful lot of time defining the success metrics. It might go something like this:

Business Leader 1: “This is my test and the goal of this test should be to increase units of Product X.”
Business Leader 2: “But without decreasing units of my Product Y”
Channel Leader: “And without shifting any sales out my Selling Channel into the Web channel.”
Etc.

Syndicate content