Repository navigation
test(hydrogen): add property tests with fast-check - #4354
Draft
fredericoo wants to merge 2 commits into
Draft
fredericoo wants to merge 2 commits into
fredericoo wants to merge 2 commits into
Conversation
Adds fast-check as a devDependency and 13 property tests across input handling, security rules, round trips, async races and action sequences. Six properties fail on main and are left red on purpose. Each one points at a bug or a rule that needs a decision; the fixes come in a stacked PR.
Contributor
|
Oxygen deployed a preview of your
Learn more about Hydrogen's GitHub integration. |
A cart property that holds runs 1,000 sessions. On CI that took 2.4 seconds, about half of Vitest's 5-second default, and three more properties will run in full once their fixes land.
Collaborator
|
Overall i think the PR makes sense adding the fast-check. PR mentions that we will stack fixes. Perhaps additionally we should scope domains worth adding fast checks to. When I did this initially, I caught some bugs on these too:
|
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TL;DR: Hydrogen's tests check the inputs and the response orders someone thought of. This PR adds fast-check, which generates them, and 13 property tests for
@shopify/hydrogen. Seven pass. Six fail onmainand stay red in this PR on purpose. Each one points at a bug or a rule that needs a decision, and the fixes come in a PR stacked on this one.A property test states a rule, such as "a settled cart matches Shopify's cart". fast-check then searches for an input, or an order of responses, that breaks the rule. When it finds one, it shrinks it to a small failing case and prints a seed that replays it.
Before
One hand-picked input per test. Nothing tried
sanitizeQuantity("1e309").After
What this changes
fast-checkas a devDependency of@shopify/hydrogen, with no configuration. Seeds stay random on every run, as fast-check recommends.*.property.test.tsfiles next to the code they test:mainsanitizeQuantityreturns a whole number of at least 1 for any inputgetSameOriginPathonly returns a location on the origin, and only for a target on the origincreateRunWithCacheserves cached data to the key that stored it, and to no other keyformatMoneyformats any price with any valid display optionsreset()fc.scheduler()decides when each response reaches the store, so every run tries a different order.AGENTS.md.The six failures
sanitizeQuantity("1e309")returnsInfinity, and so does"Infinity". When the cart store gets such a quantity in asetsubmission, the cart showsInfinityand sendsquantity: Infinity, which the request to Hydrogen's cart route serialises asnull. Hydrogen's own quantity field accepts digits only, so through that field it takes a number of at least 309 digits. A custom field or a script gets there with1e309. A cart fragment that selectsquantityAvailablecaps the quantity and avoids this."\ud800"and"\ud801", share one cache entry, and share it with the same key holding U+FFFD in their place.TextEncoderturns each unpaired surrogate into U+FFFD before hashing. Array keys are not affected. Low severity: such strings come from input likeJSON.parse('"\\ud800"').formatMoneythrows. It throwsRangeError: maximumFractionDigits value is out of rangewhen the output has no currency symbol and only one fraction-digit limit is set:maximumFractionDigitsbelow 2, orminimumFractionDigitsabove 2. The output has no currency symbol withwithoutCurrency: true, with codes that are not three letters such asUSDC, and in thewithoutTrailingZerosAndCurrencyfield. The exception is a whole amount whose trailing zeros are stripped, which formats.reset()while a background refresh is in flight, for example straight afterrefresh(), leaves the cart empty withloading: trueand never loads it again. The existing test "reset ignores an in-flight authoritative revalidation" expects exactly that, so this one needs a decision before a fix.Developer impact
No changeset. The diff adds tests and a devDependency, and nothing in the published package changes.
Out of scope
updateCartcalls from code, cart attributes, the first cart load, anddestroy().formatMoneyshows an amount of-0as-$0.00, so a range of0and-0reads differently in each order. The money generator leaves-0out.Risk