Skip to content

Speed up round-trip without changing the wire format - #84

Merged
gonzalocasas merged 6 commits into
mainfrom
perf/round-trip
Aug 10, 2026
Merged

gonzalocasas merged 6 commits into
mainfrom
perf/round-trip

Conversation

@gonzalocasas

Copy link
Copy Markdown
Member

Three small-ish fixes found by profiling large mesh/graph round-trips:

  • Move plugin discovery up (_ensure_serializers) out of the per-element any_to_pb/any_from_pb to top-level.
  • Gate the legacy Any-unpacking .Is() probes in _deserialize_any
  • In _fill_attribute_columns drop the redundant per-column sort (indices are already ascending) and the throwaway list(range(count)) dense check; in mesh_to_pb read x/y/z straight from the vertex dict and fill packed fields with a single extend instead of one proto call per vertex/face.

Mesh serialize ~2x faster, graph round-trip ~1.5x faster. Wire output is unchanged; all 71 tests pass.

What type of change is this?

  • Bug fix in a backwards-compatible manner.
  • New feature in a backwards-compatible manner.
  • Breaking change: bug fix or new feature that involve incompatible API changes.
  • Other (e.g. doc update, configuration, etc)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

  • I added a line to the CHANGELOG.md file in the Unreleased section under the most fitting heading (e.g. Added, Changed, Removed).
  • I ran all tests on my computer and it's all green (i.e. invoke test).
  • I ran lint on my computer and there are no errors (i.e. invoke lint).
  • I added new functions/classes and made them available on a second-level import, e.g. compas.datastructures.Mesh.
  • I have added tests that prove my fix is effective or that my feature works.
  • I have added necessary documentation (if appropriate)

Three targeted fixes found by profiling large mesh/graph round-trips:

- Hoist plugin discovery (_ensure_serializers) out of the per-element
  any_to_pb/any_from_pb path to the top-level entry points. It was
  idempotent but ran once per element (200k times for a 10k-node graph);
  now runs once per operation.
- Gate the legacy Any-unpacking .Is() probes in _deserialize_any behind
  the "message" field so they no longer fire on every scalar.
- In _fill_attribute_columns drop the redundant per-column sort (indices
  are already ascending) and the throwaway list(range(count)) dense check;
  in mesh_to_pb read x/y/z straight from the vertex dict and fill packed
  fields with a single extend instead of one proto call per vertex/face.

Mesh serialize ~2x faster, graph round-trip ~1.5x faster. Wire output is
unchanged; all 71 tests pass.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@gonzalocasas
gonzalocasas requested review from chenkasirer and a lite review from Copilot and removed request for Copilot August 5, 2026 11:36
gonzalocasas and others added 2 commits August 5, 2026 13:50
mesh_to_pb read x/y/z straight off the vertex attribute dicts, but
add_vertex only stores the keys it was passed, so a vertex that relies on
default_vertex_attributes for any coordinate has no entry at all and
serialization raised KeyError. Resolve the three defaults once outside the
loop and read through them, which keeps the batched extend.

The packed vertices array is the geometry and carries no defaults beside
it, so it has to hold the effective coordinates. That is the opposite of
the attribute columns, which correctly store only explicit values because
the default maps travel alongside them.

Also covers non-geometry defaults on vertices, faces and edges, including
that they stay live defaults rather than being baked into each element,
sparse columns with no default declared, and non-float attribute types.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Mesh serialization now reads coordinates from the vertex attribute dict, so
subclasses that compute them in a vertex_coordinates() override serialize the
stored values instead. Flag that explicitly along with the workarounds.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR optimizes compas_pb mesh/graph serialization and deserialization hot paths to reduce round-trip time while aiming to keep the protobuf wire format unchanged. It does so by moving one-time setup work out of per-element loops and reducing per-element protobuf API calls, with added regression tests around mesh defaults and attribute column behavior.

Changes:

  • Moves plugin discovery (_ensure_serializers) out of per-element any_to_pb/any_from_pb and into top-level serialize/deserialize entry points.
  • Optimizes _deserialize_any, _fill_attribute_columns, and mesh_to_pb to reduce redundant checks, sorting, and per-element proto calls.
  • Adds mesh round-trip tests covering default coordinates and sparse/dense/non-float attribute columns; documents behavioral implications in the changelog.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File Description
tests/test_mesh.py Adds regression tests for mesh vertex default coordinates and attribute/default preservation across round-trips.
src/compas_pb/core.py Moves serializer/plugin discovery to top-level entry points and gates legacy Any-unpacking checks.
src/compas_pb/conversions.py Optimizes attribute column building and mesh serialization (packed coordinates + batched extends).
CHANGELOG.md Notes the performance work and calls out the vertex_coordinates() override implication for subclasses.

Comment thread src/compas_pb/core.py
Comment thread src/compas_pb/core.py
gonzalocasas and others added 2 commits August 5, 2026 14:13
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated no new comments.

Suppressed comments (3)

src/compas_pb/core.py:110

  • any_to_pb only needs serializers, but it currently triggers plugin discovery when either serializers OR deserializers are missing. This can cause unnecessary plugin discovery / imports in serialization-only call paths and makes the condition stricter than required.
    if not SerializerRegistry._SERIALIZERS or not SerializerRegistry._DESERIALIZERS:
        _ensure_serializers()

src/compas_pb/core.py:142

  • any_from_pb only needs deserializers, but it currently triggers plugin discovery based on serializer availability too. This adds unnecessary work and side effects when only deserialization is needed.
    if not SerializerRegistry._SERIALIZERS or not SerializerRegistry._DESERIALIZERS:
        _ensure_serializers()

src/compas_pb/conversions.py:421

  • mesh_to_pb now iterates mesh.vertex.items() directly. This bypasses the public mesh.vertices() iterator used previously and can change vertex ordering/selection for subclasses or implementations where vertices() is overridden or ordered differently, which can also change the serialized output ordering.
    for index, (key, attr) in enumerate(mesh.vertex.items()):
        coords.append(attr.get("x", default_x))
        coords.append(attr.get("y", default_y))
        coords.append(attr.get("z", default_z))
        index_map[key] = index

@WeiTing1991 WeiTing1991 left a comment •

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! LGTM. Thanks for performance improvement. 🚀

@chenkasirer chenkasirer left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@gonzalocasas
gonzalocasas merged commit f136642 into main Aug 10, 2026
17 checks passed
@gonzalocasas
gonzalocasas deleted the perf/round-trip branch August 10, 2026 11:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants