Decoding A/B Testing: The Consequences of Continuous Monitoring
I recently watched a detailed seminar by Ramesh Johari of Stanford University (here) that delved into the practical implications of A/B testing, particularly in the context of continually monitored experiments. The focus was on how standard statistical methods may fail when applied in this dynamic scenario and how his team at Optimizely has adapted methodologies accordingly. A/B testing, or A/B experimentation, is essentially a modern incarnation of randomized controlled trials (RCTs).