feat: per-job oidc_tokens block for custom audiences

docs/docs/reference/openid.md

@@ -14,8 +14,6 @@ Semaphore generates a unique OIDC token for every job. The token is injected int
 
 The token consists of a JWT token signed by Semaphore and contains the following claims.
 
-Sure, here is the reordered list presented in a table with three columns: Claim, Description, Example.
-
 | Claim       | Description                                               | Example                              |
 |-------------|-----------------------------------------------------------|--------------------------------------|
 | iss         | The issuer of the token. The full URL of the organization | `https://<org-name>.semaphoreci.com` |
@@ -48,6 +46,50 @@ A token with the above claims is exported into jobs as the `SEMAPHORE_OIDC_TOKEN
 
 If the cloud provider is configured to accept OIDC tokens, it will receive the token, verify its signature by connecting back to `<org-url>.semaphoreci.com.well-known/jwts`, and if the token is valid, it will respond with a short-lived token for this specific job that can be used to fetch and modify cloud resources.
 
+## `oidc_tokens` block {#oidc_tokens-block}
+
+:::caution Coming soon
+
+The `oidc_tokens` block is being added in stages. The yaml schema and validation are available now; runtime injection of the named environment variables is part of a follow-up release. Pipelines using `oidc_tokens` will pass validation, but the env vars will not yet be populated until the runtime wiring lands.
+
+:::
+
+Jobs can declare additional OIDC tokens with custom audiences using a per-job `oidc_tokens:` block. This is needed for consumers that strictly verify a specific `aud` value (such as [PyPI's trusted publishers](https://docs.pypi.org/trusted-publishers/), which requires `aud="pypi"`).
+
+```yaml
+jobs:
+  - name: Publish
+    oidc_tokens:
+      <ENV_VAR_NAME>:
+        aud: <string-or-list-of-strings>
+    commands:
+      - ...
+```
+
+### Schema
+
+| Field | Required | Type | Description |
+|---|---|---|---|
+| `<ENV_VAR_NAME>` (the yaml key) | yes | string | Name of the environment variable. Must match `^[A-Z_][A-Z0-9_]*$`. The reserved name `SEMAPHORE_OIDC_TOKEN` is not allowed (it's the auto-injected default). |
+| `aud` | yes | string OR non-empty list of strings | Value of the JWT `aud` claim. A string becomes JWT `"aud": "<value>"`; a list of two or more strings becomes JWT `"aud": [<values>]` per [RFC 7519 §4.1.3](https://www.rfc-editor.org/rfc/rfc7519#section-4.1.3). A single-element list (e.g. `aud: ["pypi"]`) is normalized to a string in the JWT, per RFC 7519's convention for the single-audience case — required for consumers like PyPI that strictly verify `aud` as a string. Up to 8 audiences allowed; each up to 256 characters. |
+
+A job may declare up to 16 entries.
+
+### Behavior
+
+- Each entry in `oidc_tokens` produces one additional minted token, exposed as the named environment variable (the yaml key).
+- The default `SEMAPHORE_OIDC_TOKEN` is always still injected when the OIDC feature is enabled — `oidc_tokens` is purely additive.
+- All tokens share the same TTL and the same claim set as the default token, except for `aud` (per the request) and `jti` (always unique per token).
+
+### Validation errors
+
+| Error | Cause |
+|---|---|
+| Schema validation failure on `oidc_tokens` | Yaml key doesn't match the env var name regex, `aud` is missing or wrong shape, or extra unknown properties present |
+| Reserved name `SEMAPHORE_OIDC_TOKEN` used as a custom token key | Reserved key used as a custom token name |
+
+See the [Custom audiences](../using-semaphore/openid#custom-audiences) section of the OIDC guide for usage examples.
+
 ## See also
 
 - [Pipeline YAML](./pipeline-yaml)

docs/docs/reference/pipeline-yaml.md

@@ -1213,6 +1213,28 @@ It is not possible to have both `parallelism` and [`matrix`](#matrix-in-jobs) pr
 
 :::
 
+### oidc_tokens {#oidc-tokens-in-jobs}
+
+:::caution Coming soon
+
+The `oidc_tokens` block is being added in stages. The yaml schema and validation are available now; runtime injection of the named environment variables is part of a follow-up release. Pipelines using `oidc_tokens` will pass validation, but the env vars will not yet be populated until the runtime wiring lands.
+
+:::
+
+The `oidc_tokens` property declares additional [OpenID Connect (OIDC) tokens](./openid#oidc_tokens-block) with custom audiences, exposed as named environment variables in the job. This is required for token consumers (like PyPI's trusted publishers) that strictly verify a specific `aud` claim value.
+
+```yaml title="Example"
+jobs:
+  - name: Publish to PyPI
+    oidc_tokens:
+      PYPI_OIDC_TOKEN:
+        aud: pypi
+    commands:
+      - echo "$PYPI_OIDC_TOKEN"
+```
+
+See the [`oidc_tokens` block reference](./openid#oidc_tokens-block) for the full schema, validation rules, and limits.
+
 ## after_pipeline {#after_pipeline}
 
 Defines a set of jobs to execute when the pipeline is finished. The `after_pipeline` property is most commonly used for sending notifications, collecting test results, and submitting metrics.

docs/docs/using-semaphore/openid.md

@@ -377,6 +377,79 @@ vault kv get -field=value secret/data/production/my-secret
 </div>
 </details>
 
+## Custom audiences with `oidc_tokens` {#custom-audiences}
+
+:::caution Coming soon
+
+The `oidc_tokens` block is being added in stages. The yaml schema and validation are available now; runtime injection of the named environment variables is part of a follow-up release. Pipelines using `oidc_tokens` will pass validation, but the env vars will not yet be populated until the runtime wiring lands.
+
+:::
+
+By default, Semaphore injects `SEMAPHORE_OIDC_TOKEN` into every job with the `aud` claim set to your organization URL (`https://<org-name>.semaphoreci.com`). This works for cloud providers like AWS, Google Cloud, and HashiCorp Vault that accept any audience matching the configured identity provider.
+
+Some token consumers — including [PyPI's trusted publishers](https://docs.pypi.org/trusted-publishers/), npm trusted publishing, and other registries — require the OIDC token to carry a specific audience string (e.g. `aud: pypi`). For these cases, declare additional tokens in a per-job `oidc_tokens:` block.
+
+### Example: publishing to PyPI
+
+```yaml
+version: v1.0
+name: Publish to PyPI
+agent:
+  machine:
+    type: e1-standard-2
+    os_image: ubuntu2004
+blocks:
+  - name: Publish
+    task:
+      jobs:
+        - name: Build and upload
+          oidc_tokens:
+            PYPI_OIDC_TOKEN:
+              aud: pypi
+          commands:
+            - checkout
+            - sem-version python 3.11
+            - python -m pip install --upgrade build twine
+            - python -m build
+            - 'API_TOKEN=$(curl -fsS -X POST https://pypi.org/_/oidc/mint-token \
+                  -H "Content-Type: application/json" \
+                  -d "{\"token\":\"$PYPI_OIDC_TOKEN\"}" | jq -r .token)'
+            - 'TWINE_USERNAME=__token__ TWINE_PASSWORD=$API_TOKEN python -m twine upload dist/*'
+```
+
+### How it works
+
+- Each entry in `oidc_tokens` produces one additional minted token with the requested `aud`. The token is exposed as the named environment variable (the yaml key).
+- The default `SEMAPHORE_OIDC_TOKEN` is still injected with the org-URL audience whenever the OIDC feature is enabled for the organization — your existing AWS / Vault flows continue to work unchanged.
+- All tokens (default + `oidc_tokens` entries) share the same TTL and the same claim set, except for `aud` (per the request) and `jti` (always unique per token).
+
+### Validation rules
+
+The yaml key (the env var name) must match `^[A-Z_][A-Z0-9_]*$`. The reserved name `SEMAPHORE_OIDC_TOKEN` cannot be used as a custom token name (it's the auto-injected default).
+
+`aud` is required and must be either:
+
+- a string (becomes JWT `"aud": "<value>"`), or
+- a list of two or more strings (becomes JWT `"aud": [<values>]`, per [RFC 7519 §4.1.3](https://www.rfc-editor.org/rfc/rfc7519#section-4.1.3)). A single-element list (e.g., `aud: ["pypi"]`) is normalized to a string in the JWT, per RFC 7519's convention for the single-audience case — this is necessary for consumers like PyPI that strictly verify `aud` as a string.
+
+A pipeline that omits `aud`, uses an invalid env var name, or collides with `SEMAPHORE_OIDC_TOKEN` is rejected at submission time.
+
+The schema enforces these limits: up to 16 entries per job, each with an `aud` of up to 8 audiences, each audience up to 256 characters. Pipelines exceeding these limits are rejected at validation time.
+
+### Multiple consumers in one job
+
+You can declare multiple tokens with different audiences in the same job:
+
+```yaml
+oidc_tokens:
+  PYPI_OIDC_TOKEN:
+    aud: pypi
+  NPM_OIDC_TOKEN:
+    aud: ["npm-prod", "npm-staging"]
+  DOCKERHUB_OIDC_TOKEN:
+    aud: registry.hub.docker.com
+```
+
 ## Implementation details
 
 See the [OIDC token reference page](../reference/openid) to learn how the OIDC tokens are created and what fields are available.

plumber/ppl/lib/ppl/definition_reviser.ex

@@ -4,9 +4,17 @@ defmodule Ppl.DefinitionReviser do
   """
 
   alias Ppl.DefinitionReviser.{
-    BlocksGodfather, BlocksReviser, JobsGodfather, Task2Build,
-    TaskFileProperty, ImplicitDependency, WhenValidator,
-    MaxTimeLimitChecker, ParallelismValidator, JobMatrixValidator
+    BlocksGodfather,
+    BlocksReviser,
+    JobsGodfather,
+    Task2Build,
+    TaskFileProperty,
+    ImplicitDependency,
+    WhenValidator,
+    MaxTimeLimitChecker,
+    ParallelismValidator,
+    JobMatrixValidator,
+    OIDCTokensValidator
   }
 
   def revise_definition(definition, ppl_req) do
@@ -19,6 +27,7 @@ defmodule Ppl.DefinitionReviser do
          {:ok, definition} <- BlocksReviser.revise_blocks_definition(definition, ppl_req),
          {:ok, definition} <- ParallelismValidator.validate(definition),
          {:ok, definition} <- JobMatrixValidator.validate(definition),
-    do: ImplicitDependency.convert_to_explicit(definition)
+         {:ok, definition} <- OIDCTokensValidator.validate(definition),
+         do: ImplicitDependency.convert_to_explicit(definition)
   end
 end

plumber/ppl/lib/ppl/definition_reviser/oidc_tokens_validator.ex

@@ -0,0 +1,99 @@
+defmodule Ppl.DefinitionReviser.OIDCTokensValidator do
+  @moduledoc """
+  Validates the `oidc_tokens` block on jobs.
+
+  Schema-level validation (env var name regex, `aud` shape) happens earlier in
+  the definition validator. This module enforces semantic rules that JSON
+  Schema cannot express.
+
+  Currently it rejects the reserved key `SEMAPHORE_OIDC_TOKEN`: that name is
+  used for the auto-injected default OIDC token, so allowing it as a custom
+  token name would clobber the job's environment variable.
+  """
+
+  alias Util.ToTuple
+
+  @reserved_token_names ~w(SEMAPHORE_OIDC_TOKEN)
+
+  @type definition :: map()
+  @type error :: {:error, {:malformed, String.t()}}
+
+  @spec validate(definition) :: {:ok, definition} | error
+  def validate(definition) do
+    with {:ok, definition} <- do_validate(definition, "blocks"),
+         {:ok, definition} <- do_validate(definition, "after_pipeline") do
+      ToTuple.ok(definition)
+    else
+      {:error, _} = error -> error
+    end
+  end
+
+  defp do_validate(definition, "blocks") do
+    with {:ok, blocks} <- Map.fetch(definition, "blocks"),
+         :ok <- validate_blocks(blocks) do
+      {:ok, definition}
+    end
+  end
+
+  defp do_validate(definition, "after_pipeline") do
+    case Map.get(definition, "after_pipeline") do
+      nil ->
+        {:ok, definition}
+
+      blocks ->
+        blocks = Enum.map(blocks, &Map.put(&1, "name", "after_pipeline"))
+
+        case validate_blocks(blocks) do
+          :ok -> {:ok, definition}
+          {:error, _} = error -> error
+        end
+    end
+  end
+
+  defp validate_blocks(blocks) do
+    Enum.reduce_while(blocks, :ok, fn block, :ok ->
+      case validate_block(block) do
+        :ok -> {:cont, :ok}
+        {:error, _} = error -> {:halt, error}
+      end
+    end)
+  end
+
+  defp validate_block(block) do
+    block_name = Map.get(block, "name")
+    jobs = block |> get_in(["build", "jobs"]) |> List.wrap()
+
+    Enum.reduce_while(jobs, :ok, fn job, :ok ->
+      case validate_job(block_name, job) do
+        :ok -> {:cont, :ok}
+        {:error, _} = error -> {:halt, error}
+      end
+    end)
+  end
+
+  defp validate_job(block_name, job) do
+    case Map.get(job, "oidc_tokens") do
+      tokens when is_map(tokens) ->
+        check_reserved_names(block_name, Map.get(job, "name"), tokens)
+
+      _ ->
+        :ok
+    end
+  end
+
+  defp check_reserved_names(block_name, job_name, tokens) do
+    case Enum.find(@reserved_token_names, &Map.has_key?(tokens, &1)) do
+      nil ->
+        :ok
+
+      reserved ->
+        message = reserved_name_error(block_name, job_name, reserved)
+        {:error, {:malformed, message}}
+    end
+  end
+
+  defp reserved_name_error(block_name, job_name, name) do
+    "Token name '#{name}' is reserved and cannot be used as a custom oidc_tokens key " <>
+      "(block '#{block_name}', job '#{job_name}'). Use a different environment variable name."
+  end
+end

plumber/ppl/test/definition_reviser/global_job_config_test.exs

@@ -31,8 +31,8 @@ defmodule Ppl.DefinitionReviser.BlocksReviser.GlobalJobConfig.Test do
     Test.Support.GrpcServerHelper.setup_service_url(@url_env_name_2, project_port)
 
     on_exit(fn ->
-      System.put_env(@url_env_name, old_artifact_url)
-      System.put_env(@url_env_name_2, old_project_url)
+      restore_env(@url_env_name, old_artifact_url)
+      restore_env(@url_env_name_2, old_project_url)
     end)
 
     Test.Helpers.truncate_db()
@@ -133,4 +133,7 @@ defmodule Ppl.DefinitionReviser.BlocksReviser.GlobalJobConfig.Test do
       }
     }
   end
+
+  defp restore_env(name, nil), do: System.delete_env(name)
+  defp restore_env(name, value), do: System.put_env(name, value)
 end