Speed up round-trip without changing the wire format - #84
Merged
Merged
Conversation
Three targeted fixes found by profiling large mesh/graph round-trips: - Hoist plugin discovery (_ensure_serializers) out of the per-element any_to_pb/any_from_pb path to the top-level entry points. It was idempotent but ran once per element (200k times for a 10k-node graph); now runs once per operation. - Gate the legacy Any-unpacking .Is() probes in _deserialize_any behind the "message" field so they no longer fire on every scalar. - In _fill_attribute_columns drop the redundant per-column sort (indices are already ascending) and the throwaway list(range(count)) dense check; in mesh_to_pb read x/y/z straight from the vertex dict and fill packed fields with a single extend instead of one proto call per vertex/face. Mesh serialize ~2x faster, graph round-trip ~1.5x faster. Wire output is unchanged; all 71 tests pass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
gonzalocasas
requested review from
chenkasirer
and
a lite review from Copilot
and removed request for
Copilot
August 5, 2026 11:36
mesh_to_pb read x/y/z straight off the vertex attribute dicts, but add_vertex only stores the keys it was passed, so a vertex that relies on default_vertex_attributes for any coordinate has no entry at all and serialization raised KeyError. Resolve the three defaults once outside the loop and read through them, which keeps the batched extend. The packed vertices array is the geometry and carries no defaults beside it, so it has to hold the effective coordinates. That is the opposite of the attribute columns, which correctly store only explicit values because the default maps travel alongside them. Also covers non-geometry defaults on vertices, faces and edges, including that they stay live defaults rather than being baked into each element, sparse columns with no default declared, and non-float attribute types. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mesh serialization now reads coordinates from the vertex attribute dict, so subclasses that compute them in a vertex_coordinates() override serialize the stored values instead. Flag that explicitly along with the workarounds. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
gonzalocasas
requested review from
WeiTing1991 and
ericgozzi
and
a lite review from Copilot
August 5, 2026 11:53
Contributor
There was a problem hiding this comment.
Pull request overview
This PR optimizes compas_pb mesh/graph serialization and deserialization hot paths to reduce round-trip time while aiming to keep the protobuf wire format unchanged. It does so by moving one-time setup work out of per-element loops and reducing per-element protobuf API calls, with added regression tests around mesh defaults and attribute column behavior.
Changes:
- Moves plugin discovery (
_ensure_serializers) out of per-elementany_to_pb/any_from_pband into top-level serialize/deserialize entry points. - Optimizes
_deserialize_any,_fill_attribute_columns, andmesh_to_pbto reduce redundant checks, sorting, and per-element proto calls. - Adds mesh round-trip tests covering default coordinates and sparse/dense/non-float attribute columns; documents behavioral implications in the changelog.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.
| File | Description |
|---|---|
tests/test_mesh.py |
Adds regression tests for mesh vertex default coordinates and attribute/default preservation across round-trips. |
src/compas_pb/core.py |
Moves serializer/plugin discovery to top-level entry points and gates legacy Any-unpacking checks. |
src/compas_pb/conversions.py |
Optimizes attribute column building and mesh serialization (packed coordinates + batched extends). |
CHANGELOG.md |
Notes the performance work and calls out the vertex_coordinates() override implication for subclasses. |
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.
Suppressed comments (3)
src/compas_pb/core.py:110
any_to_pbonly needs serializers, but it currently triggers plugin discovery when either serializers OR deserializers are missing. This can cause unnecessary plugin discovery / imports in serialization-only call paths and makes the condition stricter than required.
if not SerializerRegistry._SERIALIZERS or not SerializerRegistry._DESERIALIZERS:
_ensure_serializers()
src/compas_pb/core.py:142
any_from_pbonly needs deserializers, but it currently triggers plugin discovery based on serializer availability too. This adds unnecessary work and side effects when only deserialization is needed.
if not SerializerRegistry._SERIALIZERS or not SerializerRegistry._DESERIALIZERS:
_ensure_serializers()
src/compas_pb/conversions.py:421
mesh_to_pbnow iteratesmesh.vertex.items()directly. This bypasses the publicmesh.vertices()iterator used previously and can change vertex ordering/selection for subclasses or implementations wherevertices()is overridden or ordered differently, which can also change the serialized output ordering.
for index, (key, attr) in enumerate(mesh.vertex.items()):
coords.append(attr.get("x", default_x))
coords.append(attr.get("y", default_y))
coords.append(attr.get("z", default_z))
index_map[key] = index
WeiTing1991
approved these changes
Aug 9, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three small-ish fixes found by profiling large mesh/graph round-trips:
_ensure_serializers) out of the per-elementany_to_pb/any_from_pbto top-level..Is()probes in_deserialize_any_fill_attribute_columnsdrop the redundant per-column sort (indices are already ascending) and the throwawaylist(range(count))dense check; inmesh_to_pbread x/y/z straight from the vertex dict and fill packed fields with a single extend instead of one proto call per vertex/face.Mesh serialize ~2x faster, graph round-trip ~1.5x faster. Wire output is unchanged; all 71 tests pass.
What type of change is this?
Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.CHANGELOG.mdfile in theUnreleasedsection under the most fitting heading (e.g.Added,Changed,Removed).invoke test).invoke lint).compas.datastructures.Mesh.