Currently, SPRAS does not automatically rerun post-processing steps if the logic or configuration of those steps changes. (The only time the post processing is rerun is if the algorithms change in some manner - parameters or adding/removing an algorithm). This can lead to outdated results being used without the user's awareness. This will be a problem for the continuous benchmarking.
We want to expect that if a user updates a post-processing function or its configuration, SPRAS should automatically mark dependent outputs as outdated and rerun the corresponding steps.
Proposed solution:
- Add the config file as a parameter sent to each post processing step because it is timestamped and will trigger reruns.
- Otherwise Add logic to detect changes in post-processing functions or configuration files.
Currently, SPRAS does not automatically rerun post-processing steps if the logic or configuration of those steps changes. (The only time the post processing is rerun is if the algorithms change in some manner - parameters or adding/removing an algorithm). This can lead to outdated results being used without the user's awareness. This will be a problem for the continuous benchmarking.
We want to expect that if a user updates a post-processing function or its configuration, SPRAS should automatically mark dependent outputs as outdated and rerun the corresponding steps.
Proposed solution: