Repository navigation
fix: end a dataset's turn and a vector graph's build when their work is done - #68
Merged
Merged
Conversation
…is done A dataset's turn (maxConcurrent) was released only after the last bytes of the answer were written, so a client that sent its next request at once could wait for it, and /$/stats could still show the answer running (dataset_limits failed on main at 7138bd0). It now ends once the body is read to its end, as the caller's turn already did. A vector graph was marked done before it was written to its file, so a caller that waited for the status could see the folder still written to (a_saved_graph_is_read_again_by_a_new_engine failed removing it). It now stays in the making until the file is written. The test uses tripletrail_testdir.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
Two flaky tests, both from real races in the code:
limits.rsdataset_limits(failed on main at 7138bd0, run 44707386746:runningwas 1, expected 0). A dataset's turn (maxConcurrent) was released only when the request context dropped, after the last bytes of the answer were written. A client that sends its next request as soon as an answer is in could wait for (or withmaxQueued0, be refused for) that turn, and/$/statscould still show it running. The caller's turn already ends at the end of the body (ReleaseAtEnd); the dataset turn now goes with it.vectors.rsa_saved_graph_is_read_again_by_a_new_engine(failed on PR fix(ui): show TripleTrail's plan in MatGUI's Explain tab #67, run 44720484076:remove_dir_allgaveDirectoryNotEmpty, leaving atripletrail-vector-graph-*folder). The background thread marked the graph done (building = false) before it wrote the.hnswfile, so a caller that waited for the status removed the folder whilesavewas still writing.part(andcreate_dir_allcould make it again). The graph now stays "being made" until its file is written. The test usestripletrail_testdirand checks the file is a graph with no.partleft.How it was tested
cargo test -p tripletrail --test vectors(14 passed), the saved-graph test 15 times in a row.cargo test -p tripletrail-server --test limits --test http(6 + 33 passed),dataset_limits10 times in a row.cargo fmt --check,cargo clippy -p tripletrail -p tripletrail-server --all-targets -D warnings. Notripletrail-*folders left in TMPDIR.Documentation
docs/guide/) shows it