fix: release Python GIL and isolate concurrent index searches

AGENTS.md

@@ -174,6 +174,12 @@ Use `python_bindings/bindings_common.hpp` for shared conversions. Validate array
 index state, and parameter ranges before entering the core. `py::array::forcecast` permits copies;
 do not use it where callers expect in-place mutation or pointer identity.
 
+Python index wrappers use `IndexAccess` for concurrent reads and fail-fast exclusive updates.
+Acquire and destroy its guards with the GIL held; release the GIL only inside the guard's
+lifetime, after extracting native pointers, and retain Python array owners until it is reacquired.
+Keep search parameters local to each call. C++ callers remain responsible for synchronizing
+index mutations.
+
 ### Rotation and padded dimensions
 
 Indexes quantize and search in the rotated, padded domain. FHT/Kac, RaBitQKMeans, QGKMeans,

docs/docs/index/hnsw.md

@@ -69,6 +69,11 @@ std::vector<std::vector<std::pair<float, PID>>> HierarchicalNSW::search(const fl
 -  **efSearch**: The size of the candidate set for searching HNSW base layer.
 -  **thread_num**: Number of threads to use. Each query is processed by one thread.
 
+The search window belongs to each call, so concurrent C++ searches can use
+different `efSearch` values and separate outputs while the index is unchanged.
+Python search releases the GIL. Conflicting updates raise `RuntimeError`
+immediately; see the [Python concurrency contract](../quick_start.md#threading-and-file-paths).
+
 We first pre-process the query:
 
 1. Rotate the raw query vector.  
@@ -146,8 +151,9 @@ new_ids = index.add(vectors)                       # route to the nearest centro
 index.resize(index.max_elements + 100_000)
 ```
 
-It is not safe to call `add` or `resize` while another thread searches the same
-index.
+In C++, callers must prevent `add` or `resize` from overlapping with searches
+on the same index. Python detects these conflicts and raises `RuntimeError`
+immediately.
 
 `add` prepares the result IDs, quantized vectors, and link-list allocations before
 changing the graph or point count. A failure during preparation leaves those
@@ -214,8 +220,9 @@ still walks through it and `add` may link new points to it, which keeps the grap
 connected, as in hnswlib. Search never returns it, so a query can get fewer than
 `k` results once points are removed: in Python the missing slots hold `kPidMax`
 (`2**32 - 1`) with an infinite distance. Removed points still count in
-`num_points()` and cannot be restored. It is not safe to call `remove` while
-another thread searches or adds to the same index.
+`num_points()` and cannot be restored. In C++, callers must prevent `remove`
+from overlapping with searches or updates on the same index. Python detects
+these conflicts and raises `RuntimeError` immediately.
 
 #### Choosing `ef` after removals
 

docs/docs/index/ivf.md

@@ -244,6 +244,11 @@ need separate output buffers, and must not overlap index updates.
 Python's existing `index.search(queries, k, nprobe, num_threads=4)` uses this
 batch path with the same output and precision rules.
 
+Python search releases the GIL. Concurrent searches may use independent
+`nprobe`, `high_accuracy`, and `num_threads` values. Conflicting updates raise
+`RuntimeError` immediately; see the
+[Python concurrency contract](../quick_start.md#threading-and-file-paths).
+
 ## Updating an Index
 
 Starting with 0.5.1, C++ `IVF` and Python `IvfIndex` support `add()` and `remove()`
@@ -292,8 +297,9 @@ IDs and returns the number newly removed. Duplicate IDs count once. Python
 
 The stored point count (`max_elements()` in C++, `index.max_elements` in Python)
 increases after `add()` and includes removed points. Rebuilding with C++
-`construct` or Python `build` expects that many rows. Calls to `add()` or `remove()`
-must not overlap with searches or other updates on the same index.
+`construct` or Python `build` expects that many rows. In C++, callers must prevent
+`add()` or `remove()` from overlapping with searches or other updates on the same
+index. Python detects these conflicts and raises `RuntimeError` immediately.
 
 ### Cost of `add`
 

docs/docs/index/qg.md

@@ -152,5 +152,24 @@ including empty batches. Both APIs default to one worker; `num_threads=0`
 selects the available logical CPU count, capped by the number of queries.
 Queries use the original dimension, including when the internal rotation pads it.
 
+Python search releases the GIL and uses the `ef` passed to each call without
+changing a shared default. Concurrent searches may use different windows.
+A concurrent `build()` on the same index raises `RuntimeError` immediately;
+see the [Python concurrency contract](../quick_start.md#threading-and-file-paths).
+
+The current source checkout also provides C++
+`search_batch_with_ef(queries, num_queries, k, ids, distances, ef, num_threads=1)`
+for concurrent calls with independent windows:
+
+```cpp
+qg.search_batch_with_ef(
+    queries.data(), num_queries, 10, batch_ids.data(), batch_distances.data(), 100, 4
+);
+```
+
+This method does not read or change the default set by `set_ef()`. The existing
+`search()`, `search_batch()`, and `search_with_scratch()` retain that default.
+All concurrent searches require separate outputs and an unchanged index.
+
 See `sample/cpp/symqg_indexing.cpp`, `sample/cpp/symqg_querying.cpp`, and their
 Python counterparts for complete examples.

docs/docs/quick_start.md

@@ -153,6 +153,31 @@ counts, or [QGKMeans](clustering.md#qgkmeans) for graph assignment.
 
 ### Threading and file paths
 
+The current source checkout releases the Python GIL during native index search,
+construction, updates, and file I/O. The following contract applies to
+`IvfIndex`, `HnswIndex`, and `SymqgIndex`:
+
+| Operation on the same index | Concurrent access |
+| --- | --- |
+| `search`, `search_batch`, `save`, and property reads | May run together |
+| `build`, `add`, `remove`, and `resize`, where available | Require exclusive access |
+
+A conflicting call immediately raises
+`RuntimeError("Index is busy: conflicting operation in progress")`.
+It does not wait or queue an update. Access protection includes Python input
+conversion and is released when the operation returns or raises an exception.
+Different index objects can operate independently. Concurrent saves must use
+different output paths, including any sidecar files.
+
+Search parameters such as `ef`, `nprobe`, precision, and `num_threads` belong to
+each call. When using a Python thread pool, consider `num_threads=1` to avoid
+starting multiple native worker pools. Keep input arrays and any shared backing
+storage unchanged for the entire call; do not resize or modify them from another
+thread. Returned result arrays own their storage.
+
+These access checks belong to the Python wrappers. C++ callers must synchronize
+index updates themselves and use separate search output buffers.
+
 `num_threads=0` selects the detected available logical CPU count. Positive values
 set an upper limit, capped at that count; small workloads may use fewer workers.
 Python index methods default to one thread; clustering defaults to `0`.

include/rabitqlib/index/hnsw/hnsw.hpp

@@ -182,19 +182,19 @@ class HierarchicalNSW {
    private:
     friend struct HnswPruningTestAccess;
     friend maxheap<std::pair<float, PID>> detail::search_knn_neon(
-        HierarchicalNSW&, const float*, size_t
+        HierarchicalNSW&, const float*, size_t, size_t
     );
     friend maxheap<std::pair<float, PID>> detail::search_knn_generic(
-        HierarchicalNSW&, const float*, size_t
+        HierarchicalNSW&, const float*, size_t, size_t
     );
     friend maxheap<std::pair<float, PID>> detail::search_knn_avx2(
-        HierarchicalNSW&, const float*, size_t
+        HierarchicalNSW&, const float*, size_t, size_t
     );
     friend maxheap<std::pair<float, PID>> detail::search_knn_avx512_core(
-        HierarchicalNSW&, const float*, size_t
+        HierarchicalNSW&, const float*, size_t, size_t
     );
     friend maxheap<std::pair<float, PID>> detail::search_knn_avx512_popcnt(
-        HierarchicalNSW&, const float*, size_t
+        HierarchicalNSW&, const float*, size_t, size_t
     );
 
     static constexpr PID kMaxLabelOperationLock = 65536;
@@ -400,10 +400,10 @@ class HierarchicalNSW {
     get_full_est_direct(std::vector<float>&, SplitSingleQuery<float>&, PID, HierarchicalNSW::EstimateRecord&)
         const;
 
-    maxheap<std::pair<float, PID>> search_knn(const float*, size_t);
+    maxheap<std::pair<float, PID>> search_knn(const float*, size_t, size_t);
 
     template <class Kernel>
-    maxheap<std::pair<float, PID>> search_knn_direct(const float*, size_t);
+    maxheap<std::pair<float, PID>> search_knn_direct(const float*, size_t, size_t);
 
     template <class Kernel>
     void searchBaseLayerST_AdaptiveRerankOptDirect(
@@ -1405,7 +1405,6 @@ inline void HierarchicalNSW::get_full_est_direct(
 inline std::vector<std::vector<std::pair<float, PID>>> HierarchicalNSW::search(
     const float* queries, size_t query_num, size_t TOPK, size_t efSearch, size_t thread_num
 ) {
-    set_ef(efSearch);
     std::vector<std::vector<std::pair<float, PID>>> results(query_num);
     rabitqlib::ivf::parallel_for(
         0,
@@ -1414,7 +1413,8 @@ inline std::vector<std::vector<std::pair<float, PID>>> HierarchicalNSW::search(
         [&](size_t idx, size_t /*threadId*/) {
             std::vector<float> rotated_query(padded_dim_);
             this->rotator_->rotate(queries + (idx * dim_), rotated_query.data());
-            maxheap<std::pair<float, PID>> knn = search_knn(rotated_query.data(), TOPK);
+            maxheap<std::pair<float, PID>> knn =
+                search_knn(rotated_query.data(), TOPK, std::max(efSearch, TOPK));
             while (knn.size()) {
                 results[idx].emplace_back(knn.top());
                 knn.pop();
@@ -1463,14 +1463,14 @@ inline size_t HierarchicalNSW::remove(const PID* labels, size_t n) {
 }
 
 inline maxheap<std::pair<float, PID>> HierarchicalNSW::search_knn(
-    const float* rotated_query, size_t TOPK
+    const float* rotated_query, size_t TOPK, size_t ef
 ) {
-    return detail::search_knn(*this, rotated_query, TOPK);
+    return detail::search_knn(*this, rotated_query, TOPK, ef);
 }
 
 template <class Kernel>
 inline maxheap<std::pair<float, PID>> HierarchicalNSW::search_knn_direct(
-    const float* rotated_query, size_t TOPK
+    const float* rotated_query, size_t TOPK, size_t ef
 ) {
     maxheap<std::pair<float, PID>> result;
     if (cur_element_count_ == 0) {
@@ -1545,7 +1545,7 @@ inline maxheap<std::pair<float, PID>> HierarchicalNSW::search_knn_direct(
     BoundedKNN boundedKnn(TOPK);
     searchBaseLayerST_AdaptiveRerankOptDirect<Kernel>(
         curr_obj,
-        std::max(ef_, TOPK),
+        std::max(ef, TOPK),
         TOPK,
         query_wrapper,
         q_to_centroids,

include/rabitqlib/index/symqg/qg.hpp

@@ -124,7 +124,13 @@ class QuantizedGraph<float> {
         return get_row_data(data_id);
     }
 
-    void validate_search(const float*, uint32_t, const uint32_t*, const float*) const;
+    void validate_search(const float*, uint32_t, const uint32_t*, const float*, size_t)
+        const;
+
+    void search_impl(const float*, uint32_t, uint32_t*, float*, size_t);
+
+    void
+    search_with_scratch_impl(const float*, uint32_t, uint32_t*, float*, float*, float*, float*, BatchQuery<float>&, buffer::SearchBuffer<float>&, buffer::SearchBuffer<float>&, VisitedSet&, PID, const float*);
 
     const float* prepare_build_query(
         PID,
@@ -256,6 +262,18 @@ class QuantizedGraph<float> {
         size_t num_threads = 1
     );
 
+    // Like search_batch, but ef belongs to this call and never reads or changes
+    // the default set by set_ef(). Concurrent calls may use different windows.
+    void search_batch_with_ef(
+        const float* queries,
+        size_t num_queries,
+        uint32_t knn,
+        uint32_t* results,
+        float* dists,
+        size_t ef,
+        size_t num_threads = 1
+    );
+
     /**
      * Search using caller-owned scratch buffers.
      *

include/rabitqlib/simd/hnsw_dispatch.hpp

@@ -10,26 +10,26 @@ namespace rabitqlib::hnsw {
 class HierarchicalNSW;
 namespace detail {
 std::priority_queue<std::pair<float, PID>> search_knn(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 
 std::priority_queue<std::pair<float, PID>> search_knn_neon(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 
 std::priority_queue<std::pair<float, PID>> search_knn_generic(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 std::priority_queue<std::pair<float, PID>> search_knn_avx2(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 
 std::priority_queue<std::pair<float, PID>> search_knn_avx512_core(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 
 std::priority_queue<std::pair<float, PID>> search_knn_avx512_popcnt(
-    HierarchicalNSW&, const float*, size_t
+    HierarchicalNSW&, const float*, size_t, size_t ef = 0
 );
 
 }  // namespace detail

python_bindings/bindings_common.hpp

@@ -16,6 +16,48 @@ namespace py = pybind11;
 
 namespace rabitqlib::python_bindings {
 
+// All guard acquisition/destruction happens with the GIL held. Native work may
+// release the GIL inside the guard's lifetime, but must reacquire it first on
+// exit (including exceptions). This makes conflicts fail immediately without
+// blocking the GIL or introducing a lock-order dependency.
+class IndexAccess {
+   private:
+    size_t readers_ = 0;
+    bool writing_ = false;
+
+   public:
+    class Guard {
+       private:
+        IndexAccess& access_;
+        bool writing_;
+
+       public:
+        Guard(IndexAccess& access, bool writing) : access_(access), writing_(writing) {
+            if (access.writing_ || (writing && access.readers_ != 0)) {
+                throw std::runtime_error("Index is busy: conflicting operation in progress"
+                );
+            }
+            if (writing) {
+                access.writing_ = true;
+            } else {
+                ++access.readers_;
+            }
+        }
+        Guard(const Guard&) = delete;
+        Guard& operator=(const Guard&) = delete;
+        ~Guard() {
+            if (writing_) {
+                access_.writing_ = false;
+            } else {
+                --access_.readers_;
+            }
+        }
+    };
+
+    [[nodiscard]] Guard read() { return Guard(*this, false); }
+    [[nodiscard]] Guard write() { return Guard(*this, true); }
+};
+
 inline rabitqlib::MetricType metric_from_string(const std::string& metric) {
     if (metric == "l2") {
         return rabitqlib::METRIC_L2;
@@ -85,4 +127,4 @@ ensure_1d_integer_array(py::handle value, const char* name) {
     return ensure_1d_array<int64_t>(value, name);
 }
 
-}  // namespace rabitqlib::python_bindings
+}  // namespace rabitqlib::python_bindings