Repository navigation
fix(s3): drop the whole-object checksum from ranged GET responses - #3292
Open
jayhemnani9910 wants to merge 1 commit into
Open
jayhemnani9910 wants to merge 1 commit into
jayhemnani9910 wants to merge 1 commit into
Conversation
PutObject keeps the client's x-amz-checksum-* headers and GetObject sent them back on every response, including ranged ones. boto3 validates the checksum it gets, so reading any byte range of an object it uploaded with put_object failed with FlexibleChecksumError. Leave the stored checksums out when a range is served.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary / 摘要
Reading a byte range of an object through the S3 server fails in boto3 when the object was uploaded with
put_object:PutObjectkeeps the client'sx-amz-*headers inb.meta, andGetObjectsends them back on every response, including ranged ones. The storedx-amz-checksum-crc32covers the whole object, so an SDK that validates checksums checks it against the partial body and rejects it.GetObjectnow leaves the storedX-Amz-Checksum-*entries out when it serves a range; full GETs and HEAD still return them, and other metadata is unchanged./ 此 PR 包含破坏性变更。
/ 此 PR 修改了公开 API、配置、存储格式或迁移行为。
/ 此 PR 需要关联仓库同步修改。
Related Issues / 关联 Issue
None.
Testing / 测试
go test ./...(onlypkg/aria2/rpcfails: it needs an aria2 daemon on :6800, same on main)Manual test / 手动测试:
Built main and this branch, S3 server on with a bucket mapped to a local storage, boto3 1.43:
bytes=0-4,bytes=-5andbytes=6-fail on main withFlexibleChecksumErrorand returnhello/world/worldhere; a full GET still returns the body andChecksumCRC32.A 55-request boto3 run (listing, keys with spaces and
+, metadata, copy, delete, multipart) gives the same results on main and this branch apart from the three ranged GETs.New
TestGetObjectRangeOmitsWholeObjectChecksumfails without the change;go vet ./server/s3/passes.Checklist / 检查清单
/ 我已阅读 CONTRIBUTING。
/ 我确认此贡献符合仓库许可证、贡献规范和行为准则。
gofmt,go fmt, orprettierwhere applicable./ 我已按适用情况使用
gofmt、go fmt或prettier格式化变更代码。/ 我已在适用情况下请求相关维护者或代码所有者审查。
AI Disclosure / AI 使用声明
/ 此 PR 包含 AI 辅助内容。
Tools used / 使用工具:
Usage scope / 使用范围:
Code generation / 代码生成
Tests / 测试
I have reviewed and validated all AI-assisted content included in this PR.
/ 我已审核并验证此 PR 中的所有 AI 辅助内容。