Skip to content

Added immutableCacheControl and mutableCacheControl configuration#232

Merged
emgarten merged 15 commits into
emgarten:mainfrom
hajekj:main
Mar 3, 2026
Merged

Added immutableCacheControl and mutableCacheControl configuration#232
emgarten merged 15 commits into
emgarten:mainfrom
hajekj:main

Conversation

@hajekj
Copy link
Copy Markdown
Contributor

@hajekj hajekj commented Feb 27, 2026

I opted in to implement cache configuration for files which are considered immutable - they are tied to a specific version (in their versioned folder) or a hash (pdbs) and files which are mutable (json and badges) so that we can have a little more control over how this is handled when the feed is proxied through a CDN to improve performance and optimize transfer costs.

Implemented it for both Azure and S3. The defaults are left as is - no-store.

Resolves #231

… options for Azure Storage and Amazon S3 feeds
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds configurable Cache-Control behavior for immutable vs. mutable feed artifacts to better support CDN proxy scenarios for Azure Blob Storage and Amazon S3 backends.

Changes:

  • Introduces immutableCacheControl and mutableCacheControl source settings in sleet.json and wires them through factory creation.
  • Applies the configured cache-control values during Azure blob uploads and S3 object uploads based on file type.
  • Documents the new settings and adds release notes for the new version entry.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 7 comments.

Show a summary per file
File Description
src/SleetLib/FileSystem/FileSystemFactory.cs Reads new cache-control settings from config and passes them into Azure/S3 filesystem constructors.
src/SleetLib/FileSystem/AzureFileSystem.cs Stores cache-control settings and passes them to AzureFile instances.
src/SleetLib/FileSystem/AzureFile.cs Sets BlobHttpHeaders.CacheControl per file type (immutable vs mutable) during upload.
src/SleetLib/FileSystem/AmazonS3FileSystemAbstraction.cs Extends upload API to accept a cache-control value for S3 uploads.
src/SleetLib/FileSystem/AmazonS3FileSystem.cs Stores cache-control settings and passes them into AmazonS3File.
src/SleetLib/FileSystem/AmazonS3File.cs Selects cache-control per file type and forwards it to the S3 upload abstraction.
doc/client-settings.md Documents the new caching configuration options and guidance.
ReleaseNotes.md Adds a 7.1.0 entry describing the new cache-control configuration options.
Comments suppressed due to low confidence (1)

src/SleetLib/FileSystem/AmazonS3File.cs:158

  • In the unknown file type branch, the upload still proceeds with whatever cacheControl was initialized to (currently no-cache). Since this is explicitly an unknown classification, it would be safer/more consistent to fall back to the conservative setting (e.g., no-store) rather than enabling cacheability for unclassified content.
                else
                {
                    log.LogWarning($"Unknown file type: {absoluteUri}");
                }

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread doc/client-settings.md Outdated
Comment thread doc/client-settings.md Outdated
Comment thread ReleaseNotes.md Outdated
Comment thread src/SleetLib/FileSystem/AzureFile.cs
Comment thread src/SleetLib/FileSystem/AmazonS3File.cs
Comment thread src/SleetLib/FileSystem/FileSystemFactory.cs Outdated
Comment thread src/SleetLib/FileSystem/AmazonS3File.cs Outdated
hajekj and others added 3 commits March 1, 2026 09:12
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@hajekj
Copy link
Copy Markdown
Contributor Author

hajekj commented Mar 1, 2026

I mistakenly used to no-cache instead of no-store as default. Fixed it.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 8 out of 8 changed files in this pull request and generated 4 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/SleetLib/FileSystem/FileSystemFactory.cs Outdated
Comment thread src/SleetLib/FileSystem/FileSystemFactory.cs Outdated
Comment thread src/SleetLib/FileSystem/AzureFileSystem.cs Outdated
Comment thread src/SleetLib/FileSystem/AmazonS3File.cs
hajekj and others added 4 commits March 1, 2026 09:31
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Introduce CacheControlTests for Amazon S3 and Azure backends to verify correct Cache-Control headers on uploaded blobs/files. Tests cover default ("no-store") and custom cache control values for both immutable and mutable files, using both direct file system construction and settings-based configuration. Ensures Sleet sets headers as expected for both storage providers.
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 7 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread ReleaseNotes.md
Comment thread test/Sleet.Azure.Tests/CacheControlTests.cs Outdated
Comment thread test/Sleet.AmazonS3.Tests/CacheControlTests.cs Outdated
Comment thread test/Sleet.AmazonS3.Tests/CacheControlTests.cs Outdated
Comment thread src/SleetLib/FileSystem/AzureFileSystem.cs
Comment thread src/SleetLib/FileSystem/AmazonS3FileSystem.cs
Comment thread src/SleetLib/FileSystem/AmazonS3FileSystemAbstraction.cs
hajekj and others added 3 commits March 1, 2026 09:53
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
@hajekj
Copy link
Copy Markdown
Contributor Author

hajekj commented Mar 1, 2026

@emgarten - added tests. Regarding the signature changes, do you want me to address that too?

I was able to run the Azure tests only, as I don't have any S3 account at my disposal.

Comment thread test/Sleet.AmazonS3.Tests/CacheControlTests.cs Outdated
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 10 out of 10 changed files in this pull request and generated 3 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread src/SleetLib/FileSystem/AmazonS3FileSystem.cs
Comment thread src/SleetLib/FileSystem/AmazonS3FileSystem.cs
Comment thread src/SleetLib/FileSystem/AmazonS3FileSystemAbstraction.cs
@emgarten
Copy link
Copy Markdown
Owner

emgarten commented Mar 3, 2026

@emgarten - added tests. Regarding the signature changes, do you want me to address that too?

I was able to run the Azure tests only, as I don't have any S3 account at my disposal.

The signature changes are fine. For these internal pieces I don't believe anyone is using them directly.

I ran the functional tests for S3 in your PR and pasted the failure on the line it was failing on. Take a look at that and once things pass it will be good to go. Make a fix and I'll kick off the github action again to run tests. If the fix becomes more involved I can try debugging it directly against an S3 account, but do take a look at the test output first and see if you can fix it.

I appreciate the work on this, this looks like a great improvement.

@hajekj
Copy link
Copy Markdown
Contributor Author

hajekj commented Mar 3, 2026

Okay I managed to solve it. Parsing it as CacheControlHeaderValue and comparing solves the issue. I think it's somewhere in S3 SDK that they are parsing the headers or accessing the parsed headers whereas Azure SDK returns the raw string. I think it should be ready now.

@emgarten emgarten merged commit e82c91f into emgarten:main Mar 3, 2026
6 checks passed
@emgarten
Copy link
Copy Markdown
Owner

emgarten commented Mar 3, 2026

Merged, thanks for the work on this. There will be a new 7.0.0 release of sleet soon with this change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve caching support of .nupkg files

3 participants