Skip to content

Remove Hardhat and every npm dependency - #1433

Merged
ryanio merged 16 commits into
mainfrom
ryan/remove-hardhat
Nov 10, 2026
Merged

ryanio merged 16 commits into
mainfrom
ryan/remove-hardhat

Conversation

@ryanio

@ryanio ryanio commented Nov 10, 2026 •

Copy link
Copy Markdown
Collaborator

Part 7 of the modernization series and the last Hardhat PR. After this the repo has no Node or npm dependencies: no package.json, no lockfile, nothing for a compromised package to reach.

Removed

  • The TypeScript suite: test/*.spec.ts, test/findings/, test/utils/ (about 30,000 lines).
  • Hardhat configs, tsconfig.json, config/.solcover*.js, constants/, eip-712-types/, typechain setup, and the TypeScript report scripts in scripts/.
  • package.json, yarn.lock, .nvmrc, .npmrc, .npmignore, and .gas_reports/ (Hardhat gas reporter output).
  • reference/shim/Shim.sol, which only existed because Hardhat can't compile two source folders. Foundry tests load only ReferenceConsideration and ReferenceConduitController from reference-out/, and neither depends on it.
  • Ten mocks in contracts/test/ that only the TypeScript tests used (checked by contract name and import path).
  • The four Node CI jobs (Build Artifacts, Run Tests, Run Reference Tests, Run Reference Coverage Tests). None is a required check.

What replaces the Hardhat tests

#1432 ported every case where Hardhat covered lines or branches Foundry didn't. Review here found that line coverage wasn't enough, so this PR also ports what the reviewers named (Tom's list, and Ben's point about the exploit tests):

  • PastFindings.t.sol rebuilds the four test/findings attacks. Each test fails when its fix is removed from seaport-core.
  • RevertParity.t.sol adds the aggregation, conduit, basic-order encoding, ERC-1271, EOA-zone and incrementCounter cases. It also adds explicit tests for the errors whose exact payload only the fuzz engine checked, which it skips on the reference build.
  • Contract offerers that swap an item's type, token or identifier, the transfer validation zone block, end-to-end fills through the real Seaport contract, the reference getters and TypehashDirectory.

Each revert test checks the exact error payload on both builds (both payloads where they differ) and runs the honest path too. The reply to Tom maps each gap to its tests.

Replacements for package scripts

  • scripts/plot_metrics.ts becomes scripts/plot-metrics.sh (awk, same per-call counts and percentages).
  • docs/Development.md gives the Foundry commands for fuzz metrics and the optimized build's size headroom (forge build --sizes).
  • foundry.toml no longer lists node_modules as a lib. The deployed-bytecode test still passes.

Docs

AGENTS.md, the README, docs/Development.md, Biome's excludes, lefthook's globs and .gitignore drop their Hardhat parts. AGENTS.md now says to add tools through scripts/install-tools.sh, pinned by version and SHA-256, instead of npm.

docs/Deploying.md now verifies the ConduitController with Foundry from the 1.1 tag (solc 0.8.14, via_ir, 1,000,000 runs), which rebuilds the deployed code byte for byte. The 1.7 candidates note that the deployed conduits revert with InvalidERC721TransferAmount() while seaport-types declares the uint256 form.

Stale docs are refreshed: the README lists every public audit report (Trail of Bits, both Code4rena contests, Spearbit's 1.2 review), FunctionSignatures.md is regenerated from the 1.6 build, the zone docs describe 1.6's authorizeOrder/validateOrder flow, and the 2022 Code4rena contest brief is removed.

lib/seaport-sol moves to current seaport-sol main, and scripts/install-tools.sh no longer records a checksum for a binary it failed to extract.

Verification

  • FOUNDRY_PROFILE=test forge test: 627 passed, including DeployedBytecode.t.sol.
  • FOUNDRY_PROFILE=reference MOAT_PROFILE=reference forge test: 627 passed.
  • FOUNDRY_PROFILE=offerers forge test: 45 passed.
  • scripts/lint.sh check passes, including actionlint on the trimmed workflow and the frozen-source check.
  • git grep finds no remaining reference to yarn, npm, Hardhat or typechain.

🤖 Generated with Claude Code

ryanio and others added 3 commits November 10, 2026 02:49
The Foundry suite now covers what the Hardhat suite did, so drop the
TypeScript tests, Hardhat configs, typechain setup, package.json,
yarn.lock, .nvmrc, the Node CI jobs, and the reference shim that only
existed for Hardhat. Replace scripts/plot_metrics.ts with a shell
script, document Foundry equivalents for the useful package scripts,
and stop listing node_modules as a Foundry lib.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

A maintainer should confirm that Foundry security invariants adequately replace assertions in roughly 30,000 retired test lines, since line and branch coverage parity alone does not establish equivalent regression protection.

No open issues 🎉

What changed in this PR

Completes the move to Foundry-only development following the coverage work in #1432, removing Hardhat and npm dependencies without changing deployed contract logic.

Changes:

  • Removes the legacy TypeScript tests, supporting utilities, unused mocks, and Hardhat configuration.
  • Replaces Node-based fuzz metrics reporting with a shell script and documents Foundry alternatives.
  • Removes Node CI jobs and updates documentation, lint configuration, and ignore rules.
File Description
tsconfig.json Removes TypeScript compiler configuration.
test/​utils/​types.ts Removes TypeScript order types.
test/​utils/​seeded-rng.js Removes the JavaScript seeded random generator.
test/​utils/​reports/​report_parser.ts Removes legacy gas-report parsing.
test/​utils/​reports/​comment-table.ts Removes test report formatting.
test/​utils/​helpers.ts Removes Hardhat test helpers.
test/​utils/​fixtures/​tokens.ts Removes token fixtures.
test/​utils/​fixtures/​marketplace.ts Removes marketplace and signing fixtures.
test/​utils/​fixtures/​index.ts Removes shared fixtures and assertion helpers.
test/​utils/​fixtures/​create2.ts Removes deterministic deployment fixtures.
test/​utils/​fixtures/​conduit.ts Removes conduit fixtures.
test/​utils/​faucet.ts Removes Hardhat wallet-funding helpers.
test/​utils/​events.ts Removes TypeScript event decoding.
test/​utils/​encoding.ts Removes order encoding and hashing helpers.
test/​utils/​eip712/​utils.ts Removes typed-data tree utilities.
test/​utils/​eip712/​Eip712MerkleTree.ts Removes the JavaScript bulk-order tree implementation.
test/​utils/​eip712/​defaults.ts Removes typed-data default generation.
test/​utils/​eip712/​bulk-orders.ts Removes bulk-order signing helpers.
test/​utils/​criteria.ts Removes JavaScript criteria-proof helpers.
test/​utils/​contracts.ts Removes Hardhat deployment helpers.
test/​typehashdirectory.spec.ts Retires the Hardhat typehash test.
test/​router.spec.ts Retires router tests covered by the Foundry port.
test/​getter.spec.ts Retires getter tests covered by the Foundry port.
test/​findings/​PartialFillFractionOverflow.spec.ts Removes the legacy fraction-overflow regression suite.
test/​findings/​FulfillmentOverflowWithMissingItems.spec.ts Removes the legacy fulfillment-overflow regression suite.
test/​findings/​CriteriaResolverUnhashedLeaves.spec.ts Removes the legacy criteria-leaf regression suite.
test/​findings/​AdditionalRecipientsOffByOne.spec.ts Removes the legacy additional-recipient regression suite.
scripts/​write_reports.ts Removes Hardhat gas-report generation.
scripts/​utils.ts Removes gas-report storage utilities.
scripts/​print_report.ts Removes Node-based gas-report printing.
scripts/​plot-metrics.sh Adds shell-based call counts and percentages.
scripts/​plot_metrics.ts Removes the Node-based metrics chart.
scripts/​compare_reports.ts Removes Hardhat gas-report comparisons.
scripts/​comment-table.ts Removes Node-based report formatting.
reference/​shim/​Shim.sol Removes the Hardhat multi-source compilation shim.
README.md Removes the Hardhat suite reference.
package.json Removes npm dependencies and package scripts.
lefthook.yml Removes JavaScript and TypeScript lint globs.
hardhat.config.ts Removes the primary Hardhat configuration.
hardhat-validator.config.ts Removes the Hardhat validator configuration.
hardhat-reference.config.ts Removes the reference-build Hardhat configuration.
hardhat-reference-coverage.config.ts Removes Hardhat reference coverage configuration.
hardhat-coverage.config.ts Removes Hardhat optimized-contract coverage configuration.
foundry.toml Removes node_modules from library search paths.
eip-712-types/​order.js Removes JavaScript order type definitions.
eip-712-types/​domain.js Removes JavaScript domain type definitions.
eip-712-types/​bulkOrder.js Removes JavaScript bulk-order type definitions.
docs/​Development.md Replaces Hardhat instructions with Foundry metrics and size commands.
docs/​Deploying.md Clarifies that historical verification uses the 1.1 tag’s tooling.
contracts/​test/​TestPostExecution.sol Removes the TypeScript-only post-execution zone mock.
contracts/​test/​TestInvalidZone.sol Removes the invalid-zone mock.
contracts/​test/​TestInvalidContractOffererRatifyOrder.sol Removes the invalid-ratification mock.
contracts/​test/​TestInvalidContractOfferer165.sol Removes the non-ERC165 contract-offerer mock.
contracts/​test/​TestInvalidContractOfferer.sol Removes the reverting contract-offerer mock.
contracts/​test/​TestERC721Funky.sol Removes the malformed royalty-return mock.
contracts/​test/​TestERC721Fee.sol Removes the configurable royalty mock.
contracts/​test/​TestERC1271.sol Removes the signature-validation mock.
contracts/​test/​TestBadContractOfferer.sol Removes the malformed contract-offerer response mock.
contracts/​test/​EIP1271Wallet.sol Removes the Hardhat wallet mock.
constants/​constants.js Removes JavaScript deployment constants.
config/​.solcover.js Removes Solidity coverage configuration for Hardhat.
config/​.solcover-reference.js Removes reference coverage exclusions for Hardhat.
biome.json Removes obsolete JavaScript settings and exclusions.
AGENTS.md Documents Foundry-only development and pinned standalone tools.
.nvmrc Removes the Node version pin.
.npmrc Removes npm engine configuration.
.npmignore Removes npm publishing exclusions.
.gitignore Removes obsolete Hardhat and Node ignore entries.
.github/​workflows/​test.yml Removes four Node-based build, test, and coverage jobs.
.gas_reports/​3fab586d8f77266bf79226b17cbc65cb867689e6.json Removes a historical Hardhat gas report.
.gas_reports/​10d78fe57eb635a6d8a85e34236aa29548a1b52f.json Removes a historical Hardhat gas report.

Review effort: Balanced

@ryanio

ryanio commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

On Copilot's "needs a closer look": that is the right question for reviewers, and the PR body says so. The retired Hardhat tests are covered at the line and branch level, not ported assertion for assertion. Besides #1432's 90 targeted tests, regression protection comes from the MOAT fuzz engine in test/foundry/new, which runs every fulfillment method with random orders and checks each one against expected events, balances and order status, and from running the whole Foundry suite against both the optimized and reference builds in CI. If there is a Hardhat area you want ported explicitly before this merges, name it and I will add it to #1432.

ryanio and others added 3 commits November 10, 2026 09:39
The 1.1 tag's own Hardhat setup is gone with the rest of the Node
tooling. A minimal Foundry config on that tag (solc 0.8.14, via_ir,
1,000,000 runs, london, no remappings) rebuilds the deployed runtime
code byte for byte, metadata included, and the documented init code.

Note in the 1.7 candidates that the canonical conduits revert with
InvalidERC721TransferAmount() while seaport-types declares the
uint256 form that 1.2 introduced.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@tomhirst

Copy link
Copy Markdown

I compared the deleted Hardhat specs on main with test/foundry on this branch. Most of the suite is covered, and some areas (bulk signatures, EIP-1271) are stronger than before. The cases below have no Foundry replacement, or only a partial one. Can we port them before merging, or at least track them in an issue? Most would fit in RevertParity.t.sol and run against both builds.

No replacement

  1. InvalidFulfillmentComponentData checks during aggregation (~16 tests in revert.spec.ts)
    • Hardhat tested a mismatched offerer, conduit key, item type, token, identifier or recipient, and out-of-range offer and consideration indexes, on matchOrders and fulfillAvailable(Advanced)Orders.
    • In Foundry, only the fuzz mutation mutation_invalidFulfillmentComponentData hits this error, and it only covers one case: a consideration orderIndex out of range in a match.
  2. Contract offerer returns a different item type, token or identifier (6 tests in advanced.spec.ts, offer and consideration sides)
    • testAlterOfferItem and testAlterConsiderationItem in test/foundry/offerers/AdjustedAmountOfferer.t.sol:204-250 are commented out.
    • Only a changed recipient is tested (OffererCriteriaAdvanced).
  3. Conduit ChannelClosed (3 tests in conduit.spec.ts): execute, executeWithBatch1155 and executeBatch1155 called from a closed channel. Nothing under test/ checks this error.
  4. Conduit InvalidItemType for a native item sent to a conduit (1 test). Only TransferHelper's own error is tested.
  5. InvalidBasicOrderParameterEncoding from raw calldata with bad offsets for the parameters, additional recipients or signature (3 tests). ConsiderationErrors.t.sol only calls the error helper through a wrapper, never through Seaport.
  6. InvalidCallToConduit through Seaport (1 test): ERC1155 through a conduit with ExcessReturnDataRecipient. RevertParity only uses that recipient on the conduit directly.
  7. ERC-1271 wallet revert reason passed up through Seaport (1 test, "BAD SIGNER" through fulfillBasicOrder). SignatureVerification.t.sol only tests a wallet that reverts with empty data, and EIP1271Offerer.sol still has a TODO for this.
  8. The "Zone - Transfer Validation" block in zone.spec.ts (4 tests). PausableZone.t.sol ports only the PausableZone block, so these have no direct port:
    • fulfillOrder with that zone
    • an advanced criteria order with that zone
    • a PARTIAL_RESTRICTED order with that zone
    • the 1.2 ERC1155 aggregation regression, which TestTransferValidationZoneFuzz probably covers but not exactly

Partial replacement

  1. incrementCounter: nothing checks the CounterIncremented event, checks the new counter value, or fills an order re-signed with the new counter. Only the InvalidSigner revert is tested.
  2. Overflow with a zero-amount item first (the FulfillmentOverflowWithMissingItems finding, plus 2 tests in revert.spec.ts). The plain overflow tests exist, but none puts a zero-amount item ahead of the item that overflows, which is the path that regressed.
  3. AdditionalRecipientsOffByOne finding: revertMissingOriginalConsiderationItems checks the error but not the crafted calldata that was the actual exploit.
  4. Invalid submitter on matchOrders with an EOA zone: only covered through the fuzz mutations.
  5. The Seaport contract never fills an order. BaseConsiderationTest deploys Consideration, so only the new getter tests run against Seaport itself. Nothing signs or fills an order under the "Seaport" domain separator.
  6. Getters on the reference build: getter.spec ran with REFERENCE=true, but no Foundry test calls name() or information() on referenceConsideration.
  7. TypehashDirectory: nothing compares its runtime code with independently computed typehashes for heights 1–24. It's a test helper, so the risk is low.
  8. Conduit NoContract is tested through executeBatch1155 but not executeWithBatch1155.

Weaker checks on the reference build

The Hardhat suite with REFERENCE=true checked the exact custom error for most revert cases. The fuzz engine skips the revert-reason check on reference (FuzzEngine.sol ~500-512), so it only checks that reference reverts. Errors that only the fuzz engine covers have lost error-level checking on reference:

  • the criteria resolver errors
  • ConsiderationNotMet
  • MissingFulfillmentComponentOnAggregation
  • OfferAndConsiderationRequiredOnFulfillment
  • the MismatchedFulfillment… errors
  • InexactFraction
  • PartialFillsNotEnabledForOrder
  • NoSpecifiedOrdersAvailable

Gaps 1, 5 and 6 have no test on either build now.

Suggested order

  1. Gap 1: the most branches with no test.
  2. Gaps 3 and 4: the conduit's main access check.
  3. Gap 2: re-enabling the two commented-out tests may be enough.
  4. Gaps 5, 6 and 7.
  5. One end-to-end fill against Seaport (gap 13).

The rest could go in a follow-up issue.

🤖 Generated with Claude Code

@ryanio
ryanio changed the base branch from ryan/port-hardhat-tests to main November 10, 2026 21:06
ryanio and others added 9 commits November 10, 2026 13:06
Each test rebuilds the original attack: the additional recipients
off-by-one through crafted basic order calldata, a Merkle root used as
a criteria leaf, a consideration overflow after a zero-amount item, and
a partial fill fraction that reset an order. Where the attack needs a
valid order, the honest fill is checked too.

Each test fails when its fix is removed from seaport-core: the
malicious basic order fills, the root-as-leaf fill succeeds, the
overflow check no longer fires, and the order resets to 0 filled.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Aggregation InvalidFulfillmentComponentData on match and fulfillAvailable,
conduit ChannelClosed/InvalidItemType/NoContract, basic order offset
encoding, ERC-1271 revert reasons, InvalidCallToConduit, EOA zone
submitter, incrementCounter, and exact-payload checks on both builds for
errors only the fuzz engine reached.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
seaport-sol moves to current main (5b620445): formatted, tests passing
on Foundry 1.9.0, npm dependencies corrected, the same 1.6.6/1.6.3
core and types tags this repo pins.

install-tools.sh ran tar and mv joined by &&, and set -e ignores a
failure on the left of &&, so a failed extraction during an update
kept the old binary under the new checksum. Copilot found it on
seadrop#229.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- README lists the Trail of Bits review, both Code4rena contests (1.0
  and 1.2) and Spearbit's 1.2 review.
- FunctionSignatures.md is regenerated from the 1.6 build; it listed
  1.2 and 1.1 selectors only.
- Zone docs describe 1.6: authorizeOrder before transfers and
  validateOrder after, skipped only when the zone is the caller. The
  offerer no longer bypasses the zone.
- Drops Code4rena-Guidelines.md, the 2022 contest brief.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@ryanio

ryanio commented Nov 10, 2026

Copy link
Copy Markdown
Collaborator Author

Thanks @tomhirst, this was a useful list. Everything on it is now ported in this PR, so nothing waits on a follow-up issue. Each revert test checks the exact error payload, runs on both builds unless the Hardhat original was optimized-only, and also runs the honest path so a broken fixture can't pass. The suite goes from 578 to 627 tests per build, and offerers from 39 to 45.

Gap Tests
1. InvalidFulfillmentComponentData 13 test_aggregationRejects* cases in RevertParity.t.sol. Each one runs on matchOrders, fulfillAvailableOrders and fulfillAvailableAdvancedOrders
2. Contract offerer swaps an item 6 testAlter{Offer,Consideration}{ItemType,Token,Identifier} tests in AdjustedAmountOfferer.t.sol. The commented-out tests changed amounts, which testLess*/testMore* already cover, so they're removed
3, 4, 16. Conduit ChannelClosed, InvalidItemType, NoContract via executeWithBatch1155 3 tests, each on both conduits
5, 6, 7. Basic order offsets, InvalidCallToConduit through Seaport, ERC-1271 revert reason 5 tests. The Hardhat "signature offset" case actually changed the recipients offset twice; this version changes the signature offset
8. Transfer validation zone block 4 tests in zone/TransferValidationZone.t.sol
9, 12. incrementCounter and the EOA zone submitter 2 tests: the event, the new value and a re-signed fill; and a non-zone submitter on matchOrders
10, 11. The findings PastFindings.t.sol, which rebuilds all four test/findings attacks. Each fails when its fix is removed from seaport-core
13. Fills through Seaport itself 4 tests in SeaportEndToEnd.t.sol, signed under the "Seaport" domain
14. Getters on the reference build 3 tests (TestReferenceGetters)
15. TypehashDirectory Compared with typehashes computed in the test, and with the 24 BulkOrder_Typehash_Height_* constants
Reference-build error checks 8 RevertParity tests that pin the exact error on both builds for every error you listed

Where the builds revert differently, the tests pin both payloads:

  • With a bad parameters offset, reference raises InvalidTime instead of InvalidBasicOrderParameterEncoding, because it reads the time window through the shifted struct.
  • On an EOA zone submitter, optimized raises InvalidRestrictedOrder and reference reverts with empty data.

The fuzz engine's reference skip is unchanged; the explicit tests replace what it used to cover.

The same push also bumps lib/seaport-sol to current main, fixes an install-script bug Copilot found elsewhere (tar ... && mv ... under set -e), refreshes stale docs and lists all four public audit reports in the README.

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Approval recommended

No active review issues are recorded for the migration to Foundry-only testing and tooling.

No open issues 🎉

Review effort: Balanced

@ryanio
ryanio merged commit f903de9 into main Nov 10, 2026
12 checks passed
@ryanio
ryanio deleted the ryan/remove-hardhat branch November 10, 2026 22:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants