Improving, improving and improving – that’s the only thing on the minds of developers, product managers, and marketers.
But worsening A/B tests – how about that? Oleg Yakubenko fully covers this topic in his blog.
Article contents in a nutshell (followed by our thoughts on all this)
— Improvements take so much time and money but the results are ambiguous and hypothetic
— Generally, we work on quantitative metrics – it means we deal with indexes: download speed, commission rate, prices, response time and so on. Therefore, we can change numbers and work on them both ways – decrease and increase them.
— If you think that improving some metric you improve your product or business performance… Try to make it the opposite way – make it worse for a limited number of users and draw conclusions
— Many of us unconsciously use worsening tests – and the outputs are quite useful.
And now our thoughts and several ready schemes for worsening tests
First of all, we like the whole method! It’s unusual, ground-breaking, and maybe even radical – that’s so cool!
Worsening tests from our experience:
— increase of the commission rate for a certain group of site visitors. We risked the loyalty and the return rate but revealed the dependence between the fee and the outflow of users
— radical decrease of the landing screen numbers: we learned about the interrelation of content quantity and conversion, and, as a result, about the worthwhile work effort on developing and landings scaling
— reduction in the number of posts on social media and its interrelation with site visits
In his article, the author presents examples of worsening tests based on products development. As for us, we made up some examples for business and marketing in general.
— Remove all additional extensions in context ads – specifications, shortcuts, a business card, a displayed link – and check: will CTR change? will CPC increase?
— Remove site adaptivity by setting another value of the displayed viewport area. There’s a chance that your adaptive version is worse than the full one and it will be easier for users to find necessary info without site adaptivity at all.
— Increase prices for some group of products (it’s probably one of the most obvious and common worsening tests)
Remove an expensive or additional service which looks like a standard one in the field
— Stop delivering products farther than a certain area.
As I’m writing this, Russia company “Dodo pizza” is coming to mind. My hypothesis is that choosing a limited delivery area, they manage to deliver their pizzas hot and fresh. Moreover, the delivery time is quite short – up to 60 minutes, which means a high NPS and many returning clients, a higher LTV and ARPU. All that lets them outsmart the competitors in a long run.
Did you like the article? Then subscribe to out Telegram channel