Reference architecture and CI for OGX on Red Hat OpenShift AI (RHOAI).

Work in Progress - This repository is actively evolving toward a production-ready reference architecture for OGX on RHOAI. While core functionality is operational (deployment, authentication, RAG demos), we're continuously expanding components, refining Helm charts, and adding demo scripts to showcase OGX capabilities in action.

1. Reference Architecture: Production-ready deployment of OGX using RHOAI components (VLLM, PostgreSQL, Milvus, Keycloak)
2. Automated Testing: CI that validates deployments with example client scripts
3. Integration Testing: Test RHOAI/ODH/upstream OGX images through GitHub Actions
4. Demo Scripts: Reusable examples (RAG, authentication) for downstream projects

Note: Documentation is intentionally kept minimal during early development to avoid rapid obsolescence. Use LLMs to explore the codebase and understand usage patterns.

┌─────────────────────────────────────────────────────┐
│ OGX Distribution (CRD)                               │
│  ├─ Inference: VLLM (llama-3-2-3b)                  │
│  ├─ Embeddings: VLLM (nomic-embed-text-v1.5)        │
│  ├─ Auth: Keycloak OAuth2 (RBAC + Team-based)       │
│  ├─ Vector Store: Milvus (10Gi)                     │
│  └─ Storage: PostgreSQL (5Gi)                       │
└─────────────────────────────────────────────────────┘

CI/CD: GitHub Actions workflow tests full deployment lifecycle on ROSA with configurable image overrides for testing ODH/upstream builds.

• OpenShift CLI (oc)
• Container tool (podman or docker)
• Python 3.12+
• uv

Create local config file (see values-local.yaml.example for details):

cp values-local.yaml.example values-local.yaml
# Edit values-local.yaml and set required values

./setup.sh       # Install RHOAI operator and dependencies
./provision.sh   # Deploy OGX distribution

After provisioning, credentials are read directly from K8s secrets at runtime:

# Run demos by tags (see demos/manifest.yaml for available tags)
./test.sh              # Run all demos
./test.sh simple       # Run simple demos only
./test.sh complex      # Run complex demos (requires OpenAI API key)
./test.sh rag,api      # Run demos tagged with 'rag' OR 'api'

# Available tags: simple, complex, rag, api, agents, storage, embeddings, openai-required

Or run individual demos directly:

uv run demos/rag/demo.py              # RAG with S3 file storage and vector search
uv run demos/responses/demo.py        # Multi-turn conversations with response tracking
uv run demos/responses/demo.py --prompt "What is RAG?"  # Single-turn with custom question
./demos/tests/restarttest/restarttest.sh  # Test response persistence across server restarts (requires `oc` cluster access)
uv run demos/multi_agent/demo.py      # Multi-agent research assistant

With explicit parameters:

uv run demos/rag/demo.py <OGX_URL> <KEYCLOAK_URL> <USERNAME> <PASSWORD>
uv run demos/responses/demo.py <OGX_URL> <KEYCLOAK_URL> <USERNAME> <PASSWORD>

Note: The multi-agent demo requires ogx.openaiApiKey to be set in values-local.yaml.

Run notebook demos in test mode:

./test.sh jupyter         # Run all Jupyter notebooks
./test.sh hello,jupyter   # Run notebooks tagged with 'hello' and 'jupyter'

Run interactively in your browser:

./demos/notebooks/start_notebook.sh
# Opens browser at http://localhost:8888
# Run cells with Shift+Enter

Available notebooks in demos/notebooks/:

• hello.ipynb - Simple chat completion with authentication
• rag_indexing.ipynb - Build RAG index from PDFs using Milvus vector store
• rag_inference.ipynb - Retrieval-augmented generation with question answering

Test local OGX code changes on the cluster for rapid iteration.

# 1. Clone ogx locally
git clone https://github.com/ogx-ai/ogx ~/ogx

# 2. Configure (add to values-local.yaml)
#    devLocal:
#      ogxSourcePath: ~/ogx

# 3. Deploy your changes
./deploy-local.sh
# → Builds image, pushes to in-cluster registry, restarts pod, shows logs

# 4. Test your changes
curl https://$(oc get route ogx-distribution -o jsonpath='{.spec.host}')/v1/health

# 5. Revert to official image when done
./provision.sh

Features: Uses in-cluster registry (no external accounts needed), auto-detects base image and handles authentication.

Add to values-local.yaml under devLocal: (see values-local.yaml.example):

Registry authentication fails:

REGISTRY=$(oc get route default-route -n openshift-image-registry -o jsonpath='{.spec.host}')
podman login -u $(oc whoami) -p $(oc whoami -t) --tls-verify=false $REGISTRY

Registry route not available (requires cluster-admin):

oc patch configs.imageregistry.operator.openshift.io/cluster \
  --type=merge -p '{"spec":{"defaultRoute":true}}'

Pod not using dev image:

# Check Kyverno policy exists
oc get clusterpolicy replace-rhoai-ogx-images

# Check pod image
oc get pod -l app=ogx -n redhat-ods-applications \
  -o jsonpath='{.items[0].spec.containers[0].image}'

./unprovision.sh  # Remove OGX distribution
./cleanup.sh      # Remove RHOAI operator and dependencies

CI workflow (.github/workflows/provision.yml) runs on PRs and supports image overrides:

• catalog_image: Custom RHOAI catalog source
• ogx_image: Custom OGX distro image
• operator_image: Custom operator image

This enables testing ODH/upstream builds before they're released.

Contributions welcome in:

• Additional demo scripts
• Helm chart improvements and new subcharts
• CI/CD improvements and test coverage