WMWprob is identical to the area under the curve (AUC) when using a receiver operator characteristic (ROC) analysis to summarize sensitivity vs. specificity in diagnostic testing in medicine or signal detection in human perception research and other fields. Using the oft-used example of Hanley and McNeil (1982), we cover these analyses and concepts.
WMWprob is identical to the area under the curve (AUC) when using a receiver operator characteristic (ROC) analysis to summarize sensitivity vs. specificity in diagnostic testing in medicine or signal detection in human perception research and other fields. Using the oft-used example of Hanley and McNeil (1982), we cover these analyses and concepts.
Example 1b. Regular Two-Sided CI with Tailored One-Sided Test
Traditional 95% CI, [LCL(0.025), UCL(0.025)]. Test whether the AUC exceeds the "essential" threshold of 0.80, i.e., H0: WMWprob ≤ 0.80 vs. H1: WMWprob > 0.80.
As long as subject-matter researchers pay homage to p-values, statisticians should at least give them something useful. Here, testing whether AUC in only better than chance, H0: WMWprob ≤ 0.50 vs. H1: WMWprob > 0.50, is not useful. Testing H0: WMWprob ≤ 0.80 vs. H1: WMWprob > 0.80 is somewhat arbitrary, but such thresholds are set routinely in research. e.g., "the vaccine needs to be 70% effective." To obtain a p-value for H0: WMWprob ≤ 0.80, simply add "WMWprob0 = 0.80" to the WMW() call used in Example 1a.
Ex1b <- WMW(Y=Rating, Group=TrueDiseaseStatus,
GroupLevels=c("Abnormal", "Normal"),
Alpha=c(0.025, 0.025), WMWprob0=0.80)
It is critical to know how the one-sided (directional) hypothesis structure is polarized. With WMW(), lower p-values in the table indicate less chance that the AUC (WMWprob) is less than 0.80, and thus greater chance that it exceeds.0.80. Here, p = 0.0096, which a devoted frequentist would say is strong evidence that AUC exceeds 0.80.
Hypotheses Tested
**************************************************
H0: WMWprob <= 0.80 H0: WMWodds <= 4.00
H1: WMWprob > 0.80 H1: WMWodds > 4.00
**************************************************
********************************************************************
Parameter Estimate 0.95 CI* One-Sided Hypothesis P**
....................................................................
WMWprob 0.893 [0.818, 0.940] H0: WMWprob <= 0.80 0.00960
WMWodds 8.361 [4.493, 15.559] H0: WMWodds <= 4.00 0.00960
********************************************************************
*CI error rates (alphaL, alphaU): (0.025, 0.025)
CI Method: coupling Sen (1967) & Mee (1990)
**Normal(0, 1) test statistic, Z = 2.34
P-value for H0: WMWprob >= 0.80: 1 - 0.00960 = 0.99040
Two-sided p-value (H0: WMWprob = 0.80): 0.0192
Note that a footnote provides the two-sided p-value, 2*min(p, 1–p); here, 0.0192. This tests H0: WMWprob = 0.80, but it is illogical to think that any true WMWprob could be exactly 0.800000.... On the other hand, it makes perfect sense to ask whether this AUC is below 0.80 or above 0.80. Thus, equivalently, we can perform the usual "two-sided" test by considering whether p is "significantly" near 0.00 or 1.00. Ever smaller p-values give greater support for inferring that WMWprob exceeds WMWpro0; ever larger p-values give greater support for inferring that WMWprob is less than WMWprob0. To conduct these two conjoined one-sided tests at the overall 0.05 Type I error rate, just consider whether p < 0.025 or p > 0.975. If either is true, then 2*min(p, 1–p) < 0.05. There is never a true need to report a traditional two-sided p-value.