Research Fundamentals #8
Key Features
Interpretation: A statistically significant result suggests strong evidence against the null hypothesis.
Limitations: Statistical significance does not indicate the strength or importance of an effect.
Example: A new medication reduces symptoms with p = 0.03, but the actual improvement is minimal.
Understanding Practical Significance
Key Features
Interpretation: A result can be statistically significant but practically insignificant if the effect size is too small to matter.
Effect Size Matters: Metrics like Cohen’s d, Pearson’s r, or odds ratios help quantify practical importance.
Example: A new teaching pedagogy claims to improve test scores by 1%, which is statistically significant but may not justify schoolwide budget and policy changes.
Statistical vs. Practical Significance: Key Differences
The below section combines the research skills you have learned thus far in determine quality of a research study,
Probability-based conclusion vs. Real-world impact assessment
Measure p-value, and Effect size (Cohen’s d, r, etc.)
Focus Rejection of null hypothesis Importance of effect
Decision Factor Sample size affects p-value Effect size determines meaningfulness
Why This Matters in Research
Overemphasis on p-values can lead to misleading conclusions.
Combining statistical and practical significance ensures meaningful and actionable research findings.
Researchers should report both to provide a complete interpretation of results.
Key Takeaways
Statistical significance helps determine if results are due to chance. Practical significance ensures that findings have meaningful, real-world implications. Reporting both measures provides a balanced interpretation of research results.