Feature or enhancement
Proposal:
Provide a mechanism for a library to declare that the state it keeps in context variables should be inherited by newly started threads, without requiring the whole application to enable the thread_inherit_context flag and without requiring the code that creates threads to cooperate.
Libraries are encouraged to store formerly global or thread-local state in contextvars.ContextVar so the state is safe for asyncio tasks and free-threaded code. An example is numpy's printoptions, which moved from a module-level global to a context variable so that with numpy.printoptions(...) is isolated between concurrently running tasks.
This migration has a thread-visibility regression on builds where sys.flags.thread_inherit_context is false (the default for GIL-enabled builds). New threads start with an empty context, so a binding made by the starting thread is not visible in the worker thread and the state silently reverts to its default:
with numpy.printoptions(precision=2):
# thread prints with default precision unless
# thread_inherit_context is enabled
threading.Thread(target=work).start()
When the state was a module-level global, worker threads always saw the current value. After moving to a context variable, they no longer do. This makes the migration a behavior change for existing threaded programs, which discourages libraries from adopting context variables at all.
The existing mechanisms are not usable by the library author:
-X thread_inherit_context=1 / PYTHON_THREAD_INHERIT_CONTEXT is process-wide and controlled by the application or end user, not by the library whose state is affected. A library cannot require every application embedding it to change interpreter flags, and enabling whole-context inheritance may be undesirable for unrelated reasons.threading.Thread(context=...) requires the code constructing the thread to pass the context. Threads are typically created by application code, frameworks, or thread pools that know nothing about the library's context variables.
What is missing is a middle ground between "threads inherit nothing" and "threads inherit everything": a way for a library to opt its own state into inheritance across threading.Thread.start(), on a per-library (or per-variable) basis, regardless of the process-wide flag. This would let libraries convert global state to context-local state without regressing thread behavior, and would ease the eventual transition to thread_inherit_context defaulting to true, since libraries opting in today would see no behavior change when the default flips.
Explicitly supplied contexts (Thread(context=...)) should continue to behave as they do now.
Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
https://discuss.python.org/t/enabling-thread-inherit-context-and-context-aware-warnings-by-default-on-both-builds/108205
Linked PRs
Feature or enhancement
Proposal:
Provide a mechanism for a library to declare that the state it keeps in context variables should be inherited by newly started threads, without requiring the whole application to enable the
thread_inherit_contextflag and without requiring the code that creates threads to cooperate.Libraries are encouraged to store formerly global or thread-local state in
contextvars.ContextVarso the state is safe for asyncio tasks and free-threaded code. An example is numpy's printoptions, which moved from a module-level global to a context variable so that with numpy.printoptions(...) is isolated between concurrently running tasks.This migration has a thread-visibility regression on builds where
sys.flags.thread_inherit_contextis false (the default for GIL-enabled builds). New threads start with an empty context, so a binding made by the starting thread is not visible in the worker thread and the state silently reverts to its default:When the state was a module-level global, worker threads always saw the current value. After moving to a context variable, they no longer do. This makes the migration a behavior change for existing threaded programs, which discourages libraries from adopting context variables at all.
The existing mechanisms are not usable by the library author:
-X thread_inherit_context=1/ PYTHON_THREAD_INHERIT_CONTEXT is process-wide and controlled by the application or end user, not by the library whose state is affected. A library cannot require every application embedding it to change interpreter flags, and enabling whole-context inheritance may be undesirable for unrelated reasons.threading.Thread(context=...)requires the code constructing the thread to pass the context. Threads are typically created by application code, frameworks, or thread pools that know nothing about the library's context variables.What is missing is a middle ground between "threads inherit nothing" and "threads inherit everything": a way for a library to opt its own state into inheritance across
threading.Thread.start(), on a per-library (or per-variable) basis, regardless of the process-wide flag. This would let libraries convert global state to context-local state without regressing thread behavior, and would ease the eventual transition tothread_inherit_contextdefaulting to true, since libraries opting in today would see no behavior change when the default flips.Explicitly supplied contexts (
Thread(context=...)) should continue to behave as they do now.Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
https://discuss.python.org/t/enabling-thread-inherit-context-and-context-aware-warnings-by-default-on-both-builds/108205
Linked PRs
ContextVar.thread_inheritable(). #154564