Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions docs/CONFIGURATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -294,6 +294,23 @@ Session-only and dataset-run-only scores are still sent because they are not tie

For Ruby client instances, `sample_rate` is snapshotted when the client is built. Changing `config.sample_rate` later does not update that client's score sampler or the already-initialized trace sampler. Rebuild the client with `Langfuse.reset!` when changing sampling behavior.

#### `tracing_enabled`

- **Type:** Boolean
- **Default:** `true`
- **Environment:** `LANGFUSE_TRACING_ENABLED`
- **Description:** Enables or disables Langfuse tracing and scoring

```ruby
Langfuse.configure do |config|
config.tracing_enabled = false
end
```

Explicit Ruby configuration overrides `LANGFUSE_TRACING_ENABLED`. The environment variable accepts `true` or `false` without case sensitivity. `OTEL_SDK_DISABLED=true` disables OpenTelemetry trace export. Direct score ingestion remains available.

When `tracing_enabled` is false, trace and score calls are no-ops. They do not require Langfuse credentials and do not create network requests. Buffered spans are discarded instead of exported while the setting is false. Prompt and data API calls still validate the normal client configuration when used. Synchronous `create_score!` returns `nil` while telemetry is disabled. Changes to `tracing_enabled` apply to the initialized SDK. Environment variables are read when configuration is created.

#### `logger`

- **Type:** Logger
Expand Down Expand Up @@ -804,6 +821,7 @@ Client-readiness rules:
- `base_url` must be an absolute HTTP or HTTPS URL
- `batch_size` must be a positive Integer
- `score_queue_capacity` must be a positive Integer
- `tracing_enabled` must be `true` or `false`
- `flush_interval`, `timeout`, `cache_max_size`, `cache_lock_timeout`, and `cache_refresh_threads` must be positive numbers
- `cache_ttl` must be a non-negative number
- `cache_stale_ttl` must be a non-negative number or `:indefinite`
Expand Down
22 changes: 18 additions & 4 deletions lib/langfuse.rb
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,7 @@ class UnauthorizedError < ApiError; end
require_relative "langfuse/score_value"
require_relative "langfuse/score_client"
require_relative "langfuse/prompt_renderer"
require_relative "langfuse/prompt_variables"
require_relative "langfuse/text_prompt_client"
require_relative "langfuse/chat_prompt_client"
require_relative "langfuse/timestamp_parser"
Expand Down Expand Up @@ -144,7 +145,11 @@ def client
#
# @return [Boolean] true when the local configuration is valid
def configured?
configuration.valid?
config = configuration
return config.valid? if config.telemetry_enabled?

config.validate_telemetry_disabled!
true
rescue ConfigurationError
# Reading `configuration` builds it from the environment, which can fail
# on its own before there is anything to validate.
Expand All @@ -153,7 +158,7 @@ def configured?

# Return Langfuse's internal tracer provider for explicit global OpenTelemetry installation.
#
# @return [OpenTelemetry::SDK::Trace::TracerProvider]
# @return [OpenTelemetry::SDK::Trace::TracerProvider, OpenTelemetry::Trace::TracerProvider]
# @raise [ConfigurationError] if tracing configuration is invalid
#
# @example
Expand All @@ -164,6 +169,8 @@ def configured?
#
# OpenTelemetry.tracer_provider = Langfuse.tracer_provider
def tracer_provider
return noop_tracer_provider unless configuration.trace_export_enabled?

OtelSetup.setup(configuration) unless OtelSetup.initialized?
OtelSetup.tracer_provider
rescue ConfigurationError => e
Expand Down Expand Up @@ -322,7 +329,7 @@ def create_score(name:, value:, id: nil, trace_id: nil, session_id: nil, observa
# @param data_type [Symbol] Data type (:numeric, :boolean, :categorical, :text, :correction)
# @param dataset_run_id [String, nil] Optional dataset run ID to associate with the score
# @param config_id [String, nil] Optional score config ID
# @return [String] ID of the created score
# @return [String, nil] ID of the created score, or nil when telemetry is disabled
# @raise [ArgumentError] if validation fails
# @raise [UnauthorizedError] if authentication fails
# @raise [ApiError] if the API request fails
Expand Down Expand Up @@ -441,12 +448,14 @@ def reset!
@configuration = nil
@client = nil
@noop_tracer = nil
@noop_tracer_provider = nil
@emitted_warnings = nil
rescue StandardError
# Ignore shutdown errors during reset (e.g., in tests)
@configuration = nil
@client = nil
@noop_tracer = nil
@noop_tracer_provider = nil
@emitted_warnings = nil
end

Expand Down Expand Up @@ -639,6 +648,7 @@ def wrap_otel_span(otel_span, type_str, otel_tracer, attributes: nil)
end

def ensure_tracing_started
return false unless configuration.trace_export_enabled?
Comment thread
kxzk marked this conversation as resolved.
return true if OtelSetup.initialized?

OtelSetup.setup(configuration)
Expand Down Expand Up @@ -683,7 +693,11 @@ def warning_mutex
end

def noop_tracer
@noop_tracer ||= OpenTelemetry::Trace::TracerProvider.new.tracer(LANGFUSE_TRACER_NAME, Langfuse::VERSION)
@noop_tracer ||= noop_tracer_provider.tracer(LANGFUSE_TRACER_NAME, Langfuse::VERSION)
end

def noop_tracer_provider
@noop_tracer_provider ||= OpenTelemetry::Trace::TracerProvider.new
end
end
# rubocop:enable Metrics/ClassLength
Expand Down
17 changes: 17 additions & 0 deletions lib/langfuse/chat_prompt_client.rb
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
# frozen_string_literal: true

require_relative "prompt_renderer"
require_relative "prompt_variables"

module Langfuse
# Chat prompt client for compiling chat prompts with variable substitution
Expand Down Expand Up @@ -73,6 +74,22 @@ def type
"chat"
end

# Return the unique variables referenced by all message templates
#
# Section names are included because callers must provide their values.
# Message placeholder entries are not Mustache templates and are excluded.
#
# @return [Array<String>] Referenced variable names in message and source order
# @raise [Mustache::Parser::SyntaxError] if a message contains invalid Mustache syntax
def variables
prompt.each_with_object([]) do |message, names|
normalized = symbolize_keys(message)
next if normalized[:type].to_s == PLACEHOLDER_TYPE

names.concat(PromptVariables.extract(normalized[:content] || ""))
end.uniq
end

# Compile the chat prompt with variable substitution and message placeholders
#
# Returns an array of message hashes with roles and compiled content.
Expand Down
83 changes: 55 additions & 28 deletions lib/langfuse/client.rb
Original file line number Diff line number Diff line change
Expand Up @@ -29,8 +29,13 @@ class Client
# @return [Config] The client configuration
attr_reader :config

# Return the underlying API client, building it on first non-telemetry use.
#
# @return [ApiClient] The underlying API client
attr_reader :api_client
# @raise [ConfigurationError] if the full client configuration is invalid
def api_client
@api_client || @api_client_mutex.synchronize { @api_client ||= build_validated_api_client }
end

# Pure pass-throughs to {ApiClient}. See {ApiClient} for parameter and
# return-value documentation; the public surface here is identical.
Expand Down Expand Up @@ -73,29 +78,19 @@ class Client
# @return [Client]
def initialize(config)
@config = config
@config.validate!

# Create cache if enabled
cache = create_cache if cache_enabled?

# Create API client with cache
@api_client = ApiClient.new(
public_key: config.public_key,
secret_key: config.secret_key,
base_url: config.base_url,
timeout: config.timeout,
logger: config.logger,
cache: cache,
cache_observer: config.prompt_cache_observer
)

@api_client_mutex = Mutex.new
@score_client_mutex = Mutex.new
@project_id = nil
# One-shot lookup: avoids repeated blocking API calls in URL helpers
# (trace_url, dataset_url, dataset_run_url) when the project endpoint is down.
@project_id_fetched = false

# Initialize score client for batching score events
@score_client = ScoreClient.new(api_client: @api_client, config: config)
if telemetry_enabled?
config.validate!
@api_client = build_api_client
@score_client = build_score_client
else
config.validate_telemetry_disabled!
end
end

# Fetch a prompt and return the appropriate client
Expand Down Expand Up @@ -385,7 +380,7 @@ def dataset_run_url(dataset_id:, dataset_run_id:)
# rubocop:disable Metrics/ParameterLists
def create_score(name:, value:, id: nil, trace_id: nil, session_id: nil, observation_id: nil, comment: nil,
metadata: nil, environment: nil, data_type: :numeric, dataset_run_id: nil, config_id: nil)
@score_client.create(
active_score_client&.create(
name: name,
value: value,
id: id,
Expand Down Expand Up @@ -416,7 +411,7 @@ def create_score(name:, value:, id: nil, trace_id: nil, session_id: nil, observa
# @param data_type [Symbol] Data type (:numeric, :boolean, :categorical, :text, :correction)
# @param dataset_run_id [String, nil] Optional dataset run ID to associate with the score
# @param config_id [String, nil] Optional score config ID
# @return [String] ID of the created score
# @return [String, nil] ID of the created score, or nil when telemetry is disabled
# @raise [ArgumentError] if validation fails
# @raise [UnauthorizedError] if authentication fails
# @raise [ApiError] if the API request fails
Expand All @@ -426,7 +421,7 @@ def create_score(name:, value:, id: nil, trace_id: nil, session_id: nil, observa
# rubocop:disable Metrics/ParameterLists
def create_score!(name:, value:, id: nil, trace_id: nil, session_id: nil, observation_id: nil, comment: nil,
metadata: nil, environment: nil, data_type: :numeric, dataset_run_id: nil, config_id: nil)
@score_client.create!(
active_score_client&.create!(
name: name,
value: value,
id: id,
Expand Down Expand Up @@ -460,7 +455,7 @@ def create_score!(name:, value:, id: nil, trace_id: nil, session_id: nil, observ
# client.score_active_observation(name: "accuracy", value: 0.92)
# end
def score_active_observation(name:, value:, comment: nil, metadata: nil, data_type: :numeric)
@score_client.score_active_observation(
active_score_client&.score_active_observation(
name: name,
value: value,
comment: comment,
Expand All @@ -486,7 +481,7 @@ def score_active_observation(name:, value:, comment: nil, metadata: nil, data_ty
# client.score_active_trace(name: "overall_quality", value: 5)
# end
def score_active_trace(name:, value:, comment: nil, metadata: nil, data_type: :numeric)
@score_client.score_active_trace(
active_score_client&.score_active_trace(
name: name,
value: value,
comment: comment,
Expand All @@ -504,7 +499,7 @@ def score_active_trace(name:, value:, comment: nil, metadata: nil, data_type: :n
# @example
# client.flush_scores
def flush_scores
@score_client.flush
active_score_client&.flush
end

# Shutdown the client and flush any pending scores
Expand All @@ -513,8 +508,8 @@ def flush_scores
#
# @return [void]
def shutdown
@score_client.shutdown
@api_client.shutdown
@score_client&.shutdown
@api_client_mutex.synchronize { @api_client }&.shutdown
end

# Create a new dataset
Expand Down Expand Up @@ -715,6 +710,38 @@ def run_experiment(name:, task:, data: nil, dataset_name: nil, description: nil,

private

def build_validated_api_client
config.validate!
build_api_client
end

def build_api_client
cache = create_cache if cache_enabled?
ApiClient.new(
public_key: config.public_key,
secret_key: config.secret_key,
base_url: config.base_url,
timeout: config.timeout,
logger: config.logger,
cache: cache,
cache_observer: config.prompt_cache_observer
)
end

def telemetry_enabled?
config.telemetry_enabled?
end

def active_score_client
return unless telemetry_enabled?

@score_client_mutex.synchronize { @score_client ||= build_score_client }
end

def build_score_client
ScoreClient.new(api_client: api_client, config: config)
end

attr_reader :score_client

# Build a project-scoped URL, returning nil if project ID is unavailable
Expand Down
Loading
Loading