I was in Trader Joe’s the other day. My kid was at a little drawing table they have setup with markers and blank paper. My wife was in the cashier’s line. I was watching my kid, but other than that pretty useless. So I had time to really take in all the beautiful signage in the store.

“Are all these signs handmade?” I asked the three friendly-looking employees behind the Customer Service desk.

If you were a fly on my wall, you would hear this:

Marketer: “Jared, could you get tell me how long I will need to run this test for?”
Jared: “Sure. May I inquire why you're asking?"
Marketer: “Because there are more tests I've put on the testing calendar right behind this one. I need to make sure this can finish up quickly.”

Short answer:
Not unless you have magical oompa-loompas that you don’t need to pay, and these oompa-loompas are really good at persuasion architecture, design, analytics, & web development.

Can you spot the differences?

One of the fun things I like to do when I have some free time is to try to learn from A/B tests other companies are running. My thinking is, "Why run a test when you don't have to?" There might be something you can learn and adapt to your business, or it might spark some new thinking.

Have you ever gulped down an Apple Beer? Me neither. But Matthew Wright has. He also has some great experience in A/B testing at Hewlett-Packard. So pull up a stool and join our conversation. Jared Waxman chats with Matthew Wright, Manager of HP.com Worldwide Analytics, about A/B testing within large organizations.

As part of series on A/B Testing on http://www.benchmark-analytics.com these two practitioners share their thoughts on topics including: What are the benefits? What are the misconceptions? What to look for when hiring analysts?

In any organization that does a/b testing, ideas on what to test are rarely in short supply. Test ideas seem to fly out of every corner of the company. So if there are plenty of ideas floating around, what’s the problem?

Most of them are crap shoots. Blind guesses. Shots in the dark.