Added immutableCacheControl and mutableCacheControl configuration#232
Conversation
… options for Azure Storage and Amazon S3 feeds
There was a problem hiding this comment.
Pull request overview
Adds configurable Cache-Control behavior for immutable vs. mutable feed artifacts to better support CDN proxy scenarios for Azure Blob Storage and Amazon S3 backends.
Changes:
- Introduces
immutableCacheControlandmutableCacheControlsource settings insleet.jsonand wires them through factory creation. - Applies the configured cache-control values during Azure blob uploads and S3 object uploads based on file type.
- Documents the new settings and adds release notes for the new version entry.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.
Show a summary per file
| File | Description |
|---|---|
| src/SleetLib/FileSystem/FileSystemFactory.cs | Reads new cache-control settings from config and passes them into Azure/S3 filesystem constructors. |
| src/SleetLib/FileSystem/AzureFileSystem.cs | Stores cache-control settings and passes them to AzureFile instances. |
| src/SleetLib/FileSystem/AzureFile.cs | Sets BlobHttpHeaders.CacheControl per file type (immutable vs mutable) during upload. |
| src/SleetLib/FileSystem/AmazonS3FileSystemAbstraction.cs | Extends upload API to accept a cache-control value for S3 uploads. |
| src/SleetLib/FileSystem/AmazonS3FileSystem.cs | Stores cache-control settings and passes them into AmazonS3File. |
| src/SleetLib/FileSystem/AmazonS3File.cs | Selects cache-control per file type and forwards it to the S3 upload abstraction. |
| doc/client-settings.md | Documents the new caching configuration options and guidance. |
| ReleaseNotes.md | Adds a 7.1.0 entry describing the new cache-control configuration options. |
Comments suppressed due to low confidence (1)
src/SleetLib/FileSystem/AmazonS3File.cs:158
- In the unknown file type branch, the upload still proceeds with whatever
cacheControlwas initialized to (currentlyno-cache). Since this is explicitly an unknown classification, it would be safer/more consistent to fall back to the conservative setting (e.g.,no-store) rather than enabling cacheability for unclassified content.
else
{
log.LogWarning($"Unknown file type: {absoluteUri}");
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
I mistakenly used to |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Introduce CacheControlTests for Amazon S3 and Azure backends to verify correct Cache-Control headers on uploaded blobs/files. Tests cover default ("no-store") and custom cache control values for both immutable and mutable files, using both direct file system construction and settings-based configuration. Ensures Sleet sets headers as expected for both storage providers.
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
|
@emgarten - added tests. Regarding the signature changes, do you want me to address that too? I was able to run the Azure tests only, as I don't have any S3 account at my disposal. |
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
The signature changes are fine. For these internal pieces I don't believe anyone is using them directly. I ran the functional tests for S3 in your PR and pasted the failure on the line it was failing on. Take a look at that and once things pass it will be good to go. Make a fix and I'll kick off the github action again to run tests. If the fix becomes more involved I can try debugging it directly against an S3 account, but do take a look at the test output first and see if you can fix it. I appreciate the work on this, this looks like a great improvement. |
|
Okay I managed to solve it. Parsing it as |
|
Merged, thanks for the work on this. There will be a new 7.0.0 release of sleet soon with this change. |
I opted in to implement cache configuration for files which are considered immutable - they are tied to a specific version (in their versioned folder) or a hash (pdbs) and files which are mutable (json and badges) so that we can have a little more control over how this is handled when the feed is proxied through a CDN to improve performance and optimize transfer costs.
Implemented it for both Azure and S3. The defaults are left as is -
no-store.Resolves #231