Repository navigation
Design portable fetch transport API - #693
martin-kolinek wants to merge 11 commits into
Conversation
| ```rust,ignore | ||
| pub trait Transport: Send + Sync + 'static { | ||
| fn build( | ||
| self: Box<Self>, |
There was a problem hiding this comment.
| self: Box<Self>, | |
| &self, |
why not like this? The transport should still stay boxeable
There was a problem hiding this comment.
🤖 Agreed that the trait should not expose boxing. I changed Transport to Clone + Send + Sync with validate(self, ...), and HttpClient::builder<T: Transport> now performs type erasure internally through a cloneable construction closure. Each builder build consumes a fresh transport-configuration clone; no boxed receiver is required from implementors.
There was a problem hiding this comment.
🤖 Correction after further discussion: Transport should remain dyn-compatible. The design now stores Arc<dyn Transport> internally and uses validate(self: Arc<Self>, ...), which is an object-safe receiver. HttpClient::builder<T> performs the Arc erasure internally, while runtime-selected callers may pass an erased transport directly. This avoids both Box<Self> and clone_box while allowing the factory to retain owned configuration.
There was a problem hiding this comment.
🤖 Final correction: your original &self suggestion is the cleaner contract. Transport::validate(&self, ...) is dyn-compatible and treats the transport as reusable immutable composition configuration. It returns a TransportFactory owning independent validated client state. HttpClient::builder<T> boxes internally, and builder_erased(Box<dyn Transport>) is only the explicit dynamic-construction boundary; neither Box nor Arc appears in the trait receiver.
| pub fn configure(builder: HttpClientBuilder) -> Result<HttpClient> { | ||
| builder | ||
| .connection_lifetime(LIFETIME) | ||
| .client_certificate(ClientCredentialId::new("service-client")) |
There was a problem hiding this comment.
imho, all tls specific configuration should have tls_ prefix to keep clean (reqwest did the same)
There was a problem hiding this comment.
🤖 Applied this to the broad portable builder: the methods are now tls_client_credential(...) and tls_server_name(...). I kept names concise inside TLS-specific composition builders such as fetch_hyper_rustls, where the crate/type scope already establishes that the mechanisms are TLS-specific.
Design a transport-erased HttpClientBuilder around portable library requirements, move backend mechanisms out of the public surface, and document named client credentials and TLS endpoint identity mapping. Add executable WinHTTP probes for independent routing/TLS/authority control and Nagle behavior to ground the design in measured backend capabilities. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Require every supported transport to implement the complete portable baseline, add typed pre-build extensions for deliberate backend coupling, and make unsupported certificate policies explicit portable non-goals. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Define fetch_hyper as a reusable TLS-neutral HTTP engine and move rustls/native-tls connector construction into dependency-isolated composition crates. Limit fetch stabilization to portable requirements and the typed config registry. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Rename the proposed TLS-neutral engine to fetch_hyper_common and retain fetch_hyper only when referring to the current crate being split into common and TLS composition crates. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Give portable HTTP/1.1 and HTTP/2 requirements precedence over WinHTTP's HTTP/3 preference, place response decompression in fetch, and define fallible trailer semantics. Add a direct WinHTTP probe demonstrating known- and unknown-length full-duplex HTTP/2 streaming and revise the implementation design around independent send and receive lanes. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Reconcile the portable transport design with the merged WinHTTP implementation and current fetch APIs. Model validation plus lazy materialization, clarify migration gaps, retain dependency-isolated probes, and remove accidental ring activation. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Separate target stabilization behavior from current implementation state, make transport configuration cloneable before validation, define UploadCompletion, and require platform qualification for WinHTTP duplex support. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
a6310b1 to
2a23226
Compare
Keep Transport generic and cloneable with a by-value validation method. Move boxing and type erasure behind HttpClient::builder so transport implementations do not expose boxed receivers. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Prefix portable HttpClientBuilder TLS requirements while keeping composition-scoped TLS mechanisms concise. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Store unbuilt transports as Arc<dyn Transport> and validate through an owned Arc receiver. This preserves runtime transport selection without exposing Box<Self> or clone_box APIs. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Keep Transport dyn-compatible without encoding Box or Arc ownership in its receiver. Validation borrows reusable composition configuration and returns an independently owned factory. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 0c1d47b6-8039-4748-972d-20b238499d82
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #693 +/- ##
=======================================
Coverage 100.0% 100.0%
=======================================
Files 640 639 -1
Lines 85528 85484 -44
=======================================
- Hits 85528 85484 -44
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
Martin Taillefer (geeknoid)
left a comment
There was a problem hiding this comment.
Early feedback: the target contracts currently outrun their enforcement
This is a static review of draft #693 at ba0bd5ecf38d1901f7ff5bec02aa62bf84f9f0a9 against merge base 36b9e57bd6c36779195b9989b06a36d94e78c960.
The main blockers are contract consistency: the per-origin connection cap is multiplied by isolated WinHTTP sessions; ResponseTimeout cannot cover an upload after the pipeline future returns response headers; and idle policies that the design says must be exact/rejected are silently coerced. The native full-duplex claim also needs an automated supported-Windows qualification gate, and the authority probe's prefix oracle is weaker than the exact identity contract.
No exploitable security issue was established. I reviewed all 36 unsafe occurrences in the three new WinHTTP examples (35 FFI operations/blocks and the scoped SendPtr impl); handle ownership, pointer lifetimes, directional concurrency, and joins have local safety rationale, with no proven UB or unsound safe abstraction.
Open performance decision (not inline): per-thread/per-pool-slot sessions fragment connection reuse and can repeat DNS/TLS setup. That behavior predates this diff, while the new document both calls it preferable and defers the decision to profiling. Representative handshake, reuse, scaling, and tail-latency measurements should resolve shared versus isolated sessions before stabilization.
I deduplicated against the existing inline discussion. Generated Cargo.lock was excluded. No project code, tests, build, benchmark, profiler, Miri, coverage, mutation, fuzzing, or live executable was run, and nothing was posted.
|
|
||
| - total connect deadline; | ||
| - maximum idle age, accepting values whose WinHTTP representation still implies the bound; | ||
| - total concurrent connections per origin through `WINHTTP_OPTION_MAX_CONNS_PER_SERVER`; |
There was a problem hiding this comment.
The advertised per-origin connection cap is multiplied by every isolated session — Correctness · Medium · High
WINHTTP_OPTION_MAX_CONNS_PER_SERVER is session-scoped, while implementation.md:454-492 specifies one session for every runtime thread × multiple_pools slot. Applying a requested cap of N to each session therefore permits up to N × threads × slots connections to the same origin, rather than the total cap promised here. This can exceed a caller's downstream connection budget even though validation succeeds.
Direction: enforce one client-wide permit budget per origin before dispatching into isolated sessions, or redefine the portable option explicitly as a per-partition limit and keep that semantic consistent across transports.
Done when: a multi-thread/multi-pool client cannot exceed the documented limit for one origin, or the public contract and compatibility tests intentionally specify the multiplied limit.
| than being silently dropped. A trailer frame is reached only once the body yields it, | ||
| so that failure arrives after the headers and every preceding data frame have been | ||
| sent (§7). | ||
| - **Full duplex.** For HTTP/2, response headers and body data may arrive while a known- or |
There was a problem hiding this comment.
The response deadline cannot cover an upload after execute has returned — Correctness · Medium · High
This permits response headers/body to be returned while upload continues, but §6.1 and implementation.md:1712-1716 enforce ResponseTimeout only by wrapping the HttpClient::execute pipeline future and claim that it continues through the complete upload. Once that future resolves with the response, its wrapper is gone, so it cannot time out or cancel the detached upload represented by UploadCompletion. The two contracts cannot both hold.
Direction: propagate one absolute response deadline into the shared request/upload state and make UploadCompletion enforce/cancel it, or end the response timeout at response headers and define a separate upload-completion deadline.
Done when: the documented timeout owner remains alive until every operation it promises to bound completes, with a contract test where response headers precede a stalled upload.
| The transport honors the complete portable connection contract: | ||
|
|
||
| - total connect deadline; | ||
| - maximum idle age, accepting values whose WinHTTP representation still implies the bound; |
There was a problem hiding this comment.
Unrepresentable idle policies are silently changed instead of rejected — Correctness · Medium · High
The next paragraph requires validation to reject values WinHTTP cannot honor, but implementation.md:1649-1655,1726-1730 rounds every finite value below five seconds up to five seconds and maps Unlimited to roughly 49 days. The first mapping weakens an upper-bound requirement (a requested one-second maximum may reuse for five seconds); the second is observably finite for long-running services. Both contradict the exact/monotonic requirement semantics proposed here.
Direction: reject finite bounds below WinHTTP's minimum and represent Unlimited without an idle-eviction option if possible; otherwise mark it unsupported rather than coercing it.
Done when: every accepted value preserves its stated bound/meaning, and unsupported values fail during validation with a specific error.
| | Connect deadline | Wraps connector establishment | Native resolve/connect controls with different phase boundaries | Baseline after defining one observable deadline | | ||
| | Separate resolve/send/receive timers | Not exposed by the supported Hyper path | Native controls | Transport-specific | | ||
| | Streaming request and response bodies | Supported | Native upload/download paths implemented | Required `Transport` invariant | | ||
| | Full-duplex HTTP/2 | Supported | Native behavior demonstrated on Windows 11 build 26100; directional implementation required | Required invariant; retain platform compatibility coverage | |
There was a problem hiding this comment.
A required invariant is being promoted from a single-host manual probe — Testing · Medium · High
The experiment records five manual runs on Windows 11 24H2 build 26100, while Microsoft only guarantees duplex behavior on “some versions.” fetch_winhttp/docs/design.md:68-75 requires every supported client/server baseline to pass before release, and implementation.md:1333-1341 describes future integration coverage, but this change adds only examples and no CI/release gate that executes them across the supported matrix. A later OS or runner change can therefore invalidate a required Transport invariant without failing the suite.
Direction: turn the decisive probe cases into gated integration tests and make supported-SKU qualification consume their results before advertising the capability.
Done when: every supported Windows baseline is automatically qualified (or rejected) and a duplex regression blocks release.
| authority_override | ||
| .http_authority | ||
| .as_deref() | ||
| .is_some_and(|authority| authority.starts_with(RESOLUTION_HOST)), |
There was a problem hiding this comment.
The authority probe accepts values that violate the exact-identity contract — Testing · Low · High
Both this assertion and the positive case at line 80 use starts_with, so resolution.test.invalid.attacker or an authority with the wrong port would still pass. The design requires the HTTP authority to preserve the exact logical/override host and port; this probe is intended to prove that separation, but its oracle only checks a prefix.
Direction: construct the exact expected authority, including the effective port, and compare for equality in both cases.
Done when: changing either the hostname suffix or port makes the probe fail.
|
Posted by an AI agent Human review is required at |
🤖 Defines the proposed stable API boundary for
fetchbefore implementation work begins.Summary
HttpClientBuilderaround portable library requirementsfetch_hyper_commonas a reusable TLS-neutral HTTP enginefetch_hyper_rustlsandfetch_hyper_native_tlscratesBackend evidence
:authorityTCP_NODELAYcontrolTuning policy
TCP_NODELAY