Skip to content

Use idhash as index - #523

Open
WilliamRoebuck wants to merge 10 commits into
eclipse-score:mainfrom
etas-contrib:feature/use-idhash-as-index
Open

Use idhash as index#523
WilliamRoebuck wants to merge 10 commits into
eclipse-score:mainfrom
etas-contrib:feature/use-idhash-as-index

Conversation

@WilliamRoebuck

@WilliamRoebuck WilliamRoebuck commented Aug 24, 2026

Copy link
Copy Markdown
Contributor

Resolves #463
As part of this, I have:

  • Changed DependencyGraph's value iterator to return a std::pair<const Key, T> instead of T.
  • Added an overload to componentOf for transition_UT

TODO

  • Rename GraphIndex where it occurs
  • Rename some parameters and local variables called index where identifier would be a better fit
  • Rework Transition changes. Changes are a bit hacky, maybe it should store its own map of structs rather than a map to two bitsets?
  • Rework componentOf override in transition UT. I'm not sure how it was working before, but I had to move it. Maybe it works with a smaller change now, but we should maybe just add an override for IComponent in the componentOf header

@github-actions

github-actions Bot commented Aug 24, 2026

Copy link
Copy Markdown

License Check Results

🚀 The license check job ran with the Bazel command:

bazel run --lockfile_mode=error //:license-check

Status: ⚠️ Needs Review

Click to expand output
[License Check Output]
Extracting Bazel installation...
Starting local Bazel server (8.7.0) and connecting to it...
INFO: Invocation ID: c3796c8f-78bc-4f55-baeb-50ae9539f01c
Computing main repo mapping: 
Computing main repo mapping: 
Loading: 
Loading: 0 packages loaded
Loading: 0 packages loaded
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Loading: 0 packages loaded
    currently loading: 
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)
Analyzing: target //:license-check (1 packages loaded, 0 targets configured)

Analyzing: target //:license-check (51 packages loaded, 10 targets configured)

Analyzing: target //:license-check (88 packages loaded, 10 targets configured)

Analyzing: target //:license-check (148 packages loaded, 1432 targets configured)

Analyzing: target //:license-check (156 packages loaded, 4866 targets configured)

Analyzing: target //:license-check (160 packages loaded, 9218 targets configured)

Analyzing: target //:license-check (168 packages loaded, 9417 targets configured)

Analyzing: target //:license-check (169 packages loaded, 9541 targets configured)

Analyzing: target //:license-check (173 packages loaded, 11555 targets configured)

INFO: Analyzed target //:license-check (173 packages loaded, 11555 targets configured).
INFO: Found 1 target...
Target //:license.check.license_check up-to-date:
  bazel-bin/license.check.license_check
  bazel-bin/license.check.license_check.jar
INFO: Elapsed time: 23.476s, Critical Path: 0.24s
INFO: 16 processes: 4 disk cache hit, 12 internal.
INFO: Build completed successfully, 16 total actions
INFO: Running command line: bazel-bin/license.check.license_check ./formatted.txt <args omitted>
usage: org.eclipse.dash.licenses.cli.Main [-batch <int>] [-cd <url>]
       [-confidence <int>] [-ef <url>] [-excludeSources <sources>] [-help] [-lic
       <url>] [-project <shortname>] [-repo <url>] [-review] [-summary <file>]
       [-timeout <seconds>] [-token <token>]

@WilliamRoebuck
WilliamRoebuck force-pushed the feature/use-idhash-as-index branch from 0022556 to e340109 Compare August 28, 2026 09:31
@WilliamRoebuck
WilliamRoebuck force-pushed the feature/use-idhash-as-index branch from e340109 to b5bf313 Compare August 28, 2026 09:41
@WilliamRoebuck
WilliamRoebuck force-pushed the feature/use-idhash-as-index branch from c13c457 to 21cc9f8 Compare August 28, 2026 11:08
@WilliamRoebuck
WilliamRoebuck marked this pull request as ready for review August 28, 2026 11:08
std::vector<GraphIndex> dependents;
std::vector<Key> depends_on;
std::vector<Key> dependents;
bool visited{false};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure if holding temporary info as members is a good idea, it does get rid of allocations if we had to make another data structure to hold the info thought 🤔

Would be good to also add @briefs to each and make it very clear that this is temp info.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also make sure to say that methods are not expected to reset it after use and you shall reset the values at the beginning of the method

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 45a58bc

/// before insertion (i.e. the first node is 0, second is 1, etc.).
template <typename... Args>
GraphIndex emplace(Args&&... args)
Key try_emplace(const Key& key, Args&&... args)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should add docstrings what happens if the emplace fails, what is returned.
Maybe would be better to return a Result also.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a warning and a debug assert 45a58bc

@@ -90,42 +90,45 @@ class DependencyGraph
/// reserved at construction).
std::size_t capacity() const

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add noexcept

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 45a58bc


T& operator[](GraphIndex index)
T& operator[](Key key)
{

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at() can throw here if the key isn't in the map, I think might be good to use find instead. It has the same complexity as at so there is no drawback I think.
This is for all of the other methods that use at also

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the dependency graph should validate every key passed to it, the caller already does this in cases where the identifier hash may not be present. I've added some @pre comments to methods where a missing key is invalid 45a58bc

void traverse(const Key start, PerNodeFn per_node)
{
visited.assign(visited.size(), false);
for (auto& [key, value] : nodes)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think std::for_each would look more readable here actually.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 45a58bc


/// @brief Returns the IComponent reference from an interface pointer. Useful for a template class that may take a
/// variant or a generic interface
inline IComponent& componentOf(IComponent* node)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this really needed? seems a bit unnecessary

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is for transition_UT, it defines its own componentOf method to override the std::variant one. I could leave this change out but I thought it made the UT easier to understand. Can be removed after #427 as well.

// dependencies can only be wired up once every node exists, so collect
// them while creating the nodes
std::vector<std::pair<GraphIndex, std::vector<std::string>>> pending_dependencies;
std::vector<std::pair<IdentifierHash, std::vector<std::string>>> pending_dependencies;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you actually need to store this at all. Should be able to make dependencies to nodes that are not yet in the graph. I think we verify that there is no empty dependence in the script so should be safe (do double check though)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We still can't setup dependencies before both sides are ready unfortunately, so this is still needed

// LM_LOG_DEBUG() << "Queued node " << task.component.get().getIndex() << " for "
// LM_LOG_DEBUG() << "Queued node " << task.component.get().getIdentifier() << " for "
// << (task.type == ComponentTaskType::kDeactivate ? "deactivation" : "activation")
// << " execution, jobs in progress:" << jobs_in_progress_;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe just get rid of the log?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in c3c554d

private:
/// @brief True iff @p node is a valid index into the underlying graph, i.e. in [0, size()).
bool isValidNode(GraphIndex node) const
bool isValidNode(Key node) const

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

const

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added an assertion that Key is trivially copyable so I don't think we should use a const value: c3c554d

/// successors. Detection of dependency readiness should be reworked to remove this.
std::bitset<static_cast<std::size_t>(internal::ProcessLimits::kMaxProcesses)> enqueued_set{};
/// @brief Information we need to maintain about graph nodes for the current transition
std::unordered_map<Key, NodeInfo> node_information;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

mmm why go with a hash map, bitset is nice because it won't allocate

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we can avoid using a map here, but it will only allocate during initialisation

// Sets up or resets our stored info for this transition
for (auto [key, value] : graph_)
{
state_.node_information[key] = NodeInfo{};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be better to just set the values rather than delet and construct a new element

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in c3c554d

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

Make DependencyGraph use IdentifierHash as GraphIndex

2 participants