[Pavel-Durov]

Publish ykmruby's AWFY results per merged PR to GitHub Pages, and require every PR to attach its results.

We want to see how each change affects ykmruby's performance over time and catch regressions similarly to what we have in https://ykjit.github.io/yk-benchmarks/.

Rather than copying the yklua setup, where benchmarks have to run and their results have to be collected before anything can be charted, this builds the page from the haste.zip attached to PR description. It also forces PRs to attach the benchmark evidence which is important for retrospective analysis (this is the most lightweight implementation I could think of).

High level diagram:

flowchart LR
    A[Run benchmarks] --> B[Attach haste.zip to PR description]
    B --> C[Merge PR]
    C --> D[CI builds chart from all merged PRs]
    D --> E[GitHub Pages]

Loading

Example page (only has single PR merged so far - this is the baseline)
index.html

[ltratt]

Where does this run? How do we know the results are worth looking at?

[Pavel-Durov]

Where does this run? How do we know the results are worth looking at?

The results are ran verified and added manually.
I run it on my machine and add the hate files to the Pr description.
I am doing it anyway just not in a structured way, for example in this PR #2

[ltratt]

Ah, you mean these will be published on the mruby repositories github pages?

[Pavel-Durov]

Ah, you mean these will be published on the mruby repositories github pages?

Yes, ykmruby github pages. If it works well here, I intend to do the same for yksompp and ykcsom, so we can track how changes affect performance over time.

[ltratt]

I'm generally keen to have the results somewhere. I don't want to force it to be including in PRs though: that means that only you could raise PRs, as only you have access to the consistent benchmarking machine!