Statistical Testing
Distinguish signal from chance.
Install 安装
curl -sSL https://updating.cc/skills/data/statistical-testing/SKILL.md -o ~/.agents/skills/data__statistical-testing/SKILL.mdSKILL.md Preview 技能内容预览
---
name: Statistical Testing
description: Distinguish signal from chance.
---
# Statistical Testing
Statistical tests tell you whether an observed effect is real or could plausibly be noise. Without them, every random fluctuation looks like a discovery.
## Method
- **State the hypothesis before looking at the data.** "Variant B's conversion is higher than A's." Deciding the hypothesis after seeing results is p-hacking.
- **Choose the test by data type and design.** Means → t-test/ANOVA; proportions → chi-square; paired designs → paired tests. Wrong test, wrong answer.
- **Check assumptions.** t-tests assume normality and independence; violate them and the p-value lies. Use non-parametric alternatives when assumptions fail.
- **Report effect size, not just p-value.** A statistically significant tiny effect is still tiny. "Significant" means detectable, not important.
## Avoid
- Multiple comparisons without correction. Run 20 tests at p<0.05 and one will be "significant" by chance alone. Bonferroni or FDR-correct.