ProxyAuthK8s

ProxyAuthK8S is a part of the Weebo Si project, this project is focused on exposing Kube api server with focus on security and ease of use.

In addition to the secured exposure, this project also eases the sharing of kubeconfig files between users and teams.

The base specification of the project can be found here.

Architecture

flowchart LR
  UF[Utilisateur Front] --> UI
  UC[Utilisateur CLI] --> BE

  subgraph PX[ProxyAuthK8S]
    UI[UI ProxyAuthK8S]
    BE[Backend + Controller]
    R[(Redis)]
  end

  UI --> BE
  BE <--> R
  BE <--> KC[(Cluster Kubernetes Fédérateur)]
  BE <--> IDP[Providers d'identité]
  BE --> KX[Clusters Kubernetes accessibles]

Loading

N'oublier pas de mettre une brique d'exposition entre les utilisateurs et la brique ProxyAuthK8S.

Local dev with Keycloak (alternative IdP)

You can run ProxyAuthK8S with Keycloak instead of Authelia for local OIDC testing.

1. Enable Keycloak include in compose.yaml and disable Authelia include.

2. Use the Keycloak env file.

3. Start the stack.

cp .env.keycloak .env
docker compose up -d

Keycloak URL (via Traefik): https://keycloak.k8s.localhost

The imported realm is realm-proxyauthk8s.json and already contains:

• realm: proxyauthk8s
• client proxyauthk8s (front login)
• client kube_login (cluster OIDC)
• users dev-admin / dev-user

Left to do

v0.1.0

• Create a CRD to store target cluster
• Validate the CRD against certain rules
• Authenticate users against an Oidc Provider for the UI and filter dashboard based on the user groups
• Create a UI
  • List User's accessible clusters
  • Login to cluster's OIDC if provider is OIDC and show kubeconfig
  • Show kubeconfig in case of non OIDC provider
  • Generate the Api client from the Swagger documentation of the API
• Controller
  • Reconcile CRD and update the status with the cluster accessibility for the user
  • Handle CallBack from OIDC provider and update the status
  • Handle HA of the controller with leader election
• Backend
  • Expose API for the UI
  • Redirect each request to the right cluster based on the user and the cluster accessibility
  • Validate the token either against the OIDC provider or kube itself before redirecting the request
  • Handle HA of the backend with state storage in Redis
  • Generate the Swagger documentation for the API
• CI/CD
  • On each commit
    • Run CodeQL analysis
    • Rust Deeper analysis including CVE
    • Front Lint and CVE
  • In case of TAG on main, if not ignored the tag and need to be in the format vX.X.X
    • Build and push Docker image to GHCR
    • Build and push Helm chart to GHCR
    • Create a draft-release on GitHub with the changelog
• PRE-TAG
  • Tag need to be in the format vX.X.X - Rules on repo
  • Tag need to be done with cog by a human contributor - Rules on repo
  • Each release need to have a name and a description in the changelog

v0.2.0

• Add documentation
  • How to Deploy ProxyAuthK8s
  • How to use Kubectl ProxyAuth plugin
  • How to use the UI
  • How to add new Kubernetes API to ProxyAuthK8S
  • Architecture overview
  • API documentation
  • How to contribute
  • How to setup development environment
  • How to Release a new version
• Create Krew plugin for easier usage
  • https://github.com/davidB/kubectl-view-allocations/tree/master
  • https://github.com/kubernetes-sigs/krew-index/blob/master/plugins/view-allocations.yaml
  • name: proxyauthk8s
  • https://docs.rs/clap/latest/clap/
  • Use an auto generated client from the OpenAPI spec of the API to interact with the API
• CI/CD
  • In case of TAG
    • Build the plugin archives and render the Krew manifest
    • Open the PR against the Krew Index
  • Pin actions/upload-artifact and actions/download-artifact to a SHA in release-prepare.yml, like every other action in the workflows
• Have a clean git history

v1.0.0

• Add more tests
  • Unit tests on the CRD, the path matcher and the security policy
  • Fast integration tier (wiremock upstream + real Redis)
  • envtest tier against a real ephemeral kube-apiserver
• Add redis cluster support for HA and state storage
• Add security features
  • Enforce allowed_resources on the proxy path
  • Per-cluster authorization (proxy_group)
  • Rate limiting and fail2login, backed by Redis counters
  • Audit trail on every proxied request
  • mTLS between the proxy and the target clusters
  • CEL admission rules on the CRD
  • Local JWT validation against the jwt authenticators (signature, claim validation rules and claim mappings, as the apiserver's structured authentication configuration does)
  • Allow getting oidc configuration from an external secrets
• Add Oidc token validation
• Match allowed resources on group/version/kind (AllowedCrdConfiguration) instead of only on paths
• Drop the deprecated allowed_ressources spelling once resources have been migrated to allowed_resources (breaking, hence the next major)
• CI/CD
  • In case of TAG
    • Publish the documentation on GitHub Pages

v2.0.0

• Redesign the UI (I hate the current look of it)
• Setup Exchange token between IdP and ProxyAuthK8S main auth server
• Add ability to go through a proxy (example with Netbird)
• Virtual APIs: fall back to per-namespace SelfSubjectAccessReview when LIST namespaces is denied, so LIST projects behaves like OpenShift's
• Setup Agent Mode, Allow to not expose each cluster to the world and just have an agent doing a tunnel between the Cluster ApiServer and ProxyAuthK8S