[LuYuan-Zjyy]

What does this PR do?

Closes #800.

Adds p_value() to IIDBootstrap (and, through inheritance, the block bootstraps) by inverting the confidence interval built from the same bootstrap replications. The p-value therefore matches conf_int() by construction, for every interval method.

bs = IIDBootstrap(x, seed=1234)
p = bs.p_value(func, null=0.0, reps=1000, method="basic")
p = bs.p_value(func, null=[0.0, 1.0], alternative="greater", reps=1000)

• alternative="two-sided" (default), "less", or "greater".
• null may be a scalar or one value per parameter.
• Supports basic, percentile, studentized, norm, bc, and bca interval methods.
• Reuses the interval-building logic by extracting it from conf_int() into _conf_int_from_estimates(), so no per-method p-value formula is duplicated.

Why this shape

The issue asks for the ability to obtain a p-value rather than a fixed-alpha interval. Inverting the existing interval keeps the p-value and the interval consistent automatically, and it remains valid as new interval methods are added.

Verification

• pytest arch/tests/bootstrap - 2125 passed.
• ruff check and ruff format --check on the changed files - passed.
• git diff --check - clean.
• Added tests cover two-sided inversion, one-sided alternatives, studentized with std_err_func, BCa on a scalar statistic, and argument validation.

[codecov]

Codecov Report

❌ Patch coverage is 96.09756% with 8 lines in your changes missing coverage. Please review.
✅ Project coverage is 99.49%. Comparing base (ef63ff4) to head (bbf93b1).

Files with missing lines	Patch %	Lines
arch/bootstrap/base.py	93.89%	4 Missing and 4 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1023      +/-   ##
==========================================
- Coverage   99.51%   99.49%   -0.03%     
==========================================
  Files         152      152              
  Lines       34209    34374     +165     
  Branches     2588     2610      +22     
==========================================
+ Hits        34043    34200     +157     
- Misses         94       98       +4     
- Partials       72       76       +4     

Flag	Coverage Δ
adder	99.43% <96.09%> (-0.03%)	⬇️
subtractor	99.43% <96.09%> (-0.03%)	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[bashtage]

Sounds like a good idea.

[bashtage]

Thanks for this. Inverting the interval is a good way to keep p_value consistent with conf_int. The refactor of conf_int is mechanical, and its existing tests pass.

I checked the inversion for every method. I called conf_int(..., reuse=True) after p_value, so both use the same replications. The interval at size = 1 - p then has the null on its boundary, to within 1e-17, for basic, percentile, norm, bc and bca. The two-sided percentile p-value agrees with 2 * min(F, 1 - F) from the bootstrap distribution to within 0.001.

Some things need to change:

1. A null that is NaN or None gives p = 0. np.asarray(None, dtype=float) is NaN, and NaN is never inside an interval. So p_value(func, null=np.nan) and p_value(func, null=None) both return [0., 0.], the strongest possible rejection, and null=[0.0, np.nan] returns [0.40, 0.]. _check_null should raise a TypeError for None and a ValueError for NaN, like the other argument checks.

2. null is checked only after the bootstrap has run. A null with the wrong length raises only after func has been called reps + 1 times: 501 calls with reps=500. With method="studentized" and no std_err_func, the nested bootstrap runs first as well. The type, the dimension and NaN can be checked before any computation. The length needs the number of parameters, and evaluating func once on the original data gives it before the bootstrap starts.

3. Speed. For each parameter, the search calls _conf_int_from_estimates 60 times, and each call computes the interval for every parameter. The cost therefore grows with the square of the number of parameters. With BCa, each call also recomputes the jackknife acceleration, which does not depend on size. These are times with 1,000 observations and 999 replications:

   Method, parameters	conf_int	p_value
   basic, 2	0.05 s	0.09 s
   basic, 50	0.08 s	14.2 s
   bca, 2	0.10 s	5.8 s
   bca, 10	0.12 s	36.6 s

   There are two ways to bring the cost close to that of conf_int:

   • Compute the bias and the acceleration once.
   • Search for each parameter without recomputing the others.

   The intervals can also be inverted directly. Percentile, basic, norm and studentized invert through the bootstrap distribution. BC and BCa invert through the inverse of their adjusted percentile.

4. "Matches conf_int by construction" holds only for the same replications. p_value always draws new replications and has no reuse argument, so a conf_int call followed by p_value uses different draws. With the null set to the lower bound of a 95% interval from conf_int, the one-sided p-values were 0.034 and 0.026 instead of 0.025. The Notes of the docstring and the new section of the docs say that p_value uses the same replications as conf_int. There are two options:

   • Add reuse, with the same conditions as conf_int.
   • Say that the p-value matches conf_int(..., reuse=True) called after p_value.

5. The docstring example fails as a doctest. bs.p_value(...) returns an array, but the example shows no output. Also, y comes from the global np.random.randn, which isn't seeded. pytest --doctest-modules arch/bootstrap/base.py fails on this example, and the other examples in the file pass. Either seed a Generator and show the output, or assign the result as the conf_int example does.

6. Tests. Only the percentile method is compared with conf_int. Most other tests only check that the p-values are between 0 and 1. The one-sided test puts the null 1.0 away from the estimate, so it only tells 0 apart from 1. Suggestions:

   • Parametrize the inversion test over every method and alternative. The interval at size = 1 - p should have the null on its boundary. conf_int can't reuse the replications of the studentized method, so for that method compare with a fresh instance that has the same seed.
   • Add a block bootstrap, IndependentSamplesBootstrap and a vector null.
   • Test the errors: a null that is NaN, None, a string or the wrong length, each checked before the bootstrap runs.

   codecov/patch fails because no test covers the TypeError branch of _check_null.

Smaller points:

• tail_name comes from a dict lookup, so it is a str. mypy reports Argument 2 to "_conf_int_from_estimates" of "IIDBootstrap" has incompatible type "str"; expected "Literal['two', 'upper', 'lower']". Annotating the dict fixes this.
• studentized = "studentized" in p_value is left over, since the extracted method now uses the literal.
• The p-value is exactly 0 when every replication falls on one side of the null, for example with null=10.0. The docstring should say so, and that the resolution is about 1 / reps.

[LuYuan-Zjyy]

Thanks for the detailed review. I addressed the points:

• _check_null now raises TypeError for None/non-numeric and ValueError for NaN. Type and NaN are checked before func runs; length is checked after one call to func on the original data.
• Added reuse with the same conditions as conf_int, and updated the Notes/docs so the "matches by construction" statement only applies when both share replications.
• Bias and acceleration are computed once, and each parameter is searched without rebuilding the others, so cost no longer grows quadratically. The studentized standard error now comes from the statistic replications, matching conf_int.
• The doctest example is seeded and shows output; pytest --doctest-modules arch/bootstrap/base.py passes.
• Tests now parametrize the inversion over every method and alternative, cover block and independent-samples bootstraps plus a vector null, and check None/NaN/string/wrong-length nulls before the bootstrap runs.

Local: pytest arch/tests/bootstrap - 2141 passed.

[bashtage]

Thanks.