diff --git a/.github/workflows/jira-integration.yaml b/.github/workflows/jira-integration.yaml new file mode 100644 index 0000000..34d4ee9 --- /dev/null +++ b/.github/workflows/jira-integration.yaml @@ -0,0 +1,14 @@ +name: Jira Integration +on: + pull_request: + types: [opened, edited, synchronize] + +permissions: + statuses: write + pull-requests: write + +jobs: + jira-validation: + runs-on: ubuntu-latest + steps: + - uses: mapcolonies/javascript-github-actions/actions/jira-integration@jira-integration-v1 diff --git a/README.md b/README.md index 723e544..043f0cc 100644 --- a/README.md +++ b/README.md @@ -30,7 +30,9 @@ Since we are using [Open Policy Agent](https://www.openpolicyagent.org/) (aka `O ### Log Format -The docker image provides default log format (`/etc/nginx/log_format`). It's not possible to extend the log format, so if you'd want to add/remove certain fields you have to override it. +The docker image provides a default JSON log format (`/etc/nginx/log_format.conf`). It can't be extended in place, so adding or removing fields means overriding the whole file. + +When the Fluent Bit sidecar is enabled, the chart renders a second, human-readable format alongside it: `kubectl logs` gets the readable one while the JSON format goes to the sidecar. Controlled by `fluentbit.accessLog.stdoutReadable`. ## Helm Chart @@ -46,6 +48,8 @@ There's an option to dynamically add annotations to the pod. You might find it u There's support for instrumenting NGINX with OpenTelemetry (currently only for tracing). Simply the relevant parameters in the `values.yaml` file. +There's also an optional Fluent Bit log-processing sidecar, off by default. When enabled it forwards only the selected access-log statuses and error-log severities to a central OTLP/HTTP endpoint instead of shipping every line, and derives Prometheus metrics from the access log — merged with the NGINX exporter's series onto one `/metrics` endpoint. See the `fluentbit` parameters in [values.md](./helm/values.md). + #### Overriding NGINX configuration files If you wish to override the default configuration files, you can do it by providing an external ConfigMap and supplying Volumes & VolumeMounts that'll be added to the Deployment. diff --git a/helm/config/fluent-bit.yaml b/helm/config/fluent-bit.yaml new file mode 100644 index 0000000..355446a --- /dev/null +++ b/helm/config/fluent-bit.yaml @@ -0,0 +1,199 @@ +# Fluent Bit configuration, rendered through Helm `tpl` from the chart ConfigMap. +# +# YAML, not classic `.conf`: only YAML config supports `processors`, the one way to reach an +# outgoing record's OTLP resource attributes. Classic config can set log attributes but never +# resource ones, so every forwarded record would arrive at Loki as `unknown_service`. +# +# Fluent Bit's own `fluentbit_metrics` input is deliberately not collected — too noisy. +{{- $debug := .Values.fluentbit.debug }} +{{- $stages := ternary $debug.stages (dict) $debug.enabled }} +{{- include "nginx.fluentbit.luaCallsGuard" . }} + +service: + flush: 1 + daemon: off + log_level: {{ $debug.enabled | ternary $debug.logLevel "info" }} + # Relative to this file, i.e. the image's own /fluent-bit/etc/parsers.conf (json, syslog, …). + parsers_file: parsers.conf +{{- if .Values.fluentbit.errorLog.enabled }} + +# Parses the standard open-source nginx error-log line into structured fields. Format: +# 2024/01/15 10:23:45 [error] 1234#5678: *90 , client: …, server: …, request: "…" +# The leading timestamp is optional (some syslog paths strip it) and the connection id (*cid) is +# optional (worker/startup messages omit it). nginx always emits `client:`/`server:` as a pair, +# but `request:` only once the request line has been read — so it is nested one level deeper, +# keeping client/server on lines logged before that point ("client closed connection …", SSL +# handshake failures) and on stream-module errors, which have no request at all. +# +# The tail is matched as one unit ending at `$`, with only nginx's own trailing fields (host, +# referrer, upstream, subrequest, bytes) allowed to follow. Both anchors matter: without the `$` +# a comma inside truncates it, and without the trailing whitelist an attacker-supplied +# URI echoed into (`open() "/x, client: evil, server: fake" failed`) is picked up as +# the client/server captures, because the lazy stops at the first tail that matches. +parsers: + - name: nginx_error + format: regex + regex: '^(?:(?