Analytics – Self Learning A/B test

Most companies today spend time doing A/B testing to optimize the site for better conversion.  However,  Steve Hanov has a better way to approach this problem in his latest blog post.

Decision optimization is the next step in A/B testing.  Setting some choices (Steve uses white/blue/orange colors but it could be anything) and letting the best choice determined by including randomization option in display.

def choose():
    if math.random() < 0.1:
        # exploration!
        # choose a random lever 10% of the time.
    else:
        # exploitation!
        # for each lever, 
            # calculate the expectation of reward. 
            # This is the number of trials of the lever divided by the total reward 
            # given by that lever.
        # choose the lever with the greatest expectation of reward.
    # increment the number of times the chosen lever has been played.
    # store test data in redis, choice in session key, etc..

def reward(choice, amount):
    # add the reward to the total for the given lever.

 

 

 

Advertisements
This entry was posted in Analytics and tagged . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s