Note: The job is a remote job and is open to candidates in USA. Discovered MENA is seeking a Senior Data Scientist fluent in English and Mandarin to lead their experimentation program. The role involves designing and validating statistical frameworks, ensuring the integrity of experiments, and collaborating with cross-functional teams on advanced methodologies.
Responsibilities
- Design and maintain automated anomaly detection for live experiments, including Sample Ratio Mismatch (SRM) checks and traffic split validation
- Define alerting thresholds and circuit-breaking criteria to catch and stop compromised experiments before they pollute downstream decisions
- Define and validate critical guardrail metrics (ensuring sensitivity, directionality, and interpretability) to protect business health during every release
- Implement and iterate on CUPED and related pre-experiment covariate adjustment techniques to significantly reduce metric variance
- Develop sophisticated methods to remove noise from historical user behavior, enabling faster detection of true treatment effects, especially in limited-traffic or high-priority scenarios
- Drive strategies to shorten our average experiment duration from 14 days to 9 days, unlocking a 40%+ increase in experiment velocity per quarter
- Design and run continuous, always-on A/A experiments to validate the health of our entire data pipeline (from client-side event reporting through message queues to our real-time data warehouse)
- Monitor metric baseline volatility and pipeline stability to ensure our instrumentation layer remains fully trustworthy over time. (You define what 'healthy' looks like; our platform team builds the infrastructure to monitor it)
- Partner closely with Product, Engineering, and Growth teams on advanced experiment design, sample size calculations, metric selection, and interpretation
- Advise on non-traditional frameworks (e.g., DID, synthetic control, RDD) when true randomization is not feasible
- Translate your statistical methodologies into crisp technical specifications, validation scripts, and clear decision frameworks, empowering product teams to self-serve with confidence
Skills
- Fluent in English & Mandarin
- Education: MS or PhD in Statistics, Biostatistics, Economics (Econometrics), Computer Science, or a heavily quantitative field
- Industry Experience: 3+ years of hands-on experience designing and analyzing online controlled experiments at a tech company operating at meaningful user scale (experience must go beyond survey experiments or clinical trials)
- Core Stats: Deep theoretical and practical foundations in hypothesis testing, power analysis, multiple testing correction, and sequential testing
- Specialized Methods: Proven experience implementing at least two of the following: variance reduction methods (like CUPED), sample ratio mismatch (SRM) detection, or continuous data quality monitoring for large-scale experiments
- Programming: Highly proficient in Python (scipy, statsmodels, or equivalent) for statistical modeling and simulations
- Advanced SQL mastery (window functions, complex CTEs) for robust data extraction and validation
Company Overview