Beyond conventional P-values: Addressing statistical challenges in big data
| dc.contributor.author | Zhang, Jing | |
| dc.contributor.supervisor | Zhang, Xuekui | |
| dc.contributor.supervisor | Tsao, Min | |
| dc.date.accessioned | 2026-01-23T21:46:10Z | |
| dc.date.available | 2026-01-23T21:46:10Z | |
| dc.date.issued | 2026 | |
| dc.degree.department | Department of Mathematics and Statistics | |
| dc.degree.level | Master of Science MSc | |
| dc.description.abstract | Do larger sample sizes lead to higher false positive rates in statistical analysis? The answer provided by ChatGPT 4o is ’no’, which is a common opinion shared by many statisticians. However, empirical evidence from large datasets analyses, such as those from biobanks and single-cell genomics, challenges this conclusion. Com- mon practice assesses both p-values and effect sizes to mitigate the risk of identifying spurious effects in large samples. Nonetheless, the need to adjust p-values in these contexts is unaddressed, which motivated this investigation. We found that common beliefs and practices are incorrect in real-world data analysis, since theoretical assumptions are always violated. Growing sample sizes can amplify violation impacts, inflating false positive rates. Using a simulation study, we provide examples to support our statement and illustrate a permutation-based remedy. This work’s intended contribution is to heighten awareness within our community about the pressing need to reevaluate standard statistical methods in analyzing datasets with huge sample sizes, thereby inspiring further substantial efforts to tackle this emerging challenge of the big data era. | |
| dc.description.scholarlevel | Graduate | |
| dc.identifier.uri | https://hdl.handle.net/1828/23071 | |
| dc.language | English | eng |
| dc.language.iso | en | |
| dc.rights | Available to the World Wide Web | |
| dc.subject | Big data | |
| dc.subject | Hypothesis testing | |
| dc.subject | Inflated type I error | |
| dc.subject | Violated model assumptions | |
| dc.title | Beyond conventional P-values: Addressing statistical challenges in big data | |
| dc.type | Thesis |