[Merged by Bors] - Add transform hierarchy stress test

[CptPotato]

Objective

There recently was a discussion on Discord about a possible test case for stress-testing transform hierarchies.

Solution

Create a test case for stress testing transform propagation.

Edit: I have scrapped my previous example and built something more functional and less focused on visuals.

There are three test setups:

• TestCase::Tree recursively creates a tree with a specified depth and branch width

• TestCase::NonUniformTree is the same as Tree but omits nodes in a way that makes the tree "lean" towards one side, like this:

  

• TestCase::Humanoids creates one or more separate hierarchies based on the structure of common humanoid rigs

  • this can both insert active and inactive instances of the human rig

It's possible to parameterize which parts of the hierarchy get updated (transform change) and which remain unchanged. This is based on @james7132 suggestion:
There's a probability to decide which entities should remain static. On top of that these changes can be limited to a certain range in the hierarchy (min_depth..max_depth).

[superdump]

Personally I would like to have a home for these kinds of test examples that are useful for development.

[alice-i-cecile]

Personally I would like to have a home for these kinds of test examples that are useful for development.

Seconding this: @CptPotato can you start a "stress_tests" folder in "examples" for this to live in? We can move the related benchmarky-examples into there in a follow-up PR.

@james7132 @superdump ping me when you think this is ready; I can merge this in for you to help unblock more experiments.

[james7132]

A few requests from me:

• The ability to set the proportion of how many are mutated every frame. This can be used to tell how effective our hierarchy dirtying detection/propagation is. Randomly choosing whether we include or exclude the Rotate component is sufficient. Tentatively, this should probably could also have a control on the minimum and maximum depths this occurs at.
• Non uniform distributions of hierarchy depths as an optional parameter. We'll probably find a few cases where there are huge numbers of uniform depth transforms, then a few deep cases (thinking active character models among a scene of otherwise normal scene geometry that might get de-parented/marked static in final builds).

This should let us test this at varying conditions typically expected out of a game scene (mostly static? mostly moving? how deep are the mutations?)

[CptPotato]

Good points @james7132. I think I'll also change the code to be more "functional" rather than trying to make it look nice.

I have some ideas for how to address these things. Tough, it might take me a couple of days to get to it.

[CptPotato]

Alright, I created the stress_test sub folder and moved the example there.

I have mostly rewritten the example, ditching the focus on visuals and taking some feedback into account, there's some more info in the PR description. The branch name doesn't really fit anymore but I guess that's okay?

I'm still not too sure about the configuration part. I tried making it as versatile as reasonably possible but it seems a little bloated to me.

[james7132]

One typo, and please fix the CI, but otherwise amazing. This is exactly the kind of tests we needed.

[CptPotato]

Good to hear this is on the right track.

Should we include a number of preset configurations for easier and more consistent testing?
Like:

cargo r --example transform_hierarchy -- 1
cargo r --example transform_hierarchy -- 2
...

[superdump]

If we do that, it would be good if they had memorable and descriptive names. 🙂 Like max-depth (a single long chain of hierarchy for worst case), max-width (like the many_cubes — sphere case of 160k root entities with transforms), varied-depth (perhaps specifying a depth range and the branches have paths that are various lengths within that depth range to try to simulate something reasonably realistic)?

[CptPotato]

Sorry for the delay. I added a few test cases that can be run like this:

cargo r --example transform_hierarchy -- <name>

These are the setups I came up with:

Configuration	Description
large_tree	A fairly wide and deep tree.
wide_tree	A shallow but very wide tree.
deep_tree	A deep but not very wide tree.
chain	A chain. 2500 levels deep.
update_leaves	Same as large_tree, but only leaves are updated.
update_shallow	Same as large_tree, but only the first few levels are updated.
humanoids_active	4000 active humanoid rigs.
humanoids_inactive	4000 humanoid rigs. Only 10 are active.
humanoids_mixed	2000 inactive and 2000 active humanoid rigs.
drawing	A rendering-heavy configuration. Meshes and materials are shared.
drawing_unique	Same as drawing, but all meshes and materials are unique.

I shot for ~250k to 350k entities for each of those, I think this should provide a heavy enough load.
The main exception is the chain test case. I couldn't make this one much more extreme without running into stack overflows.

---

Edit: I'm thinking about removing the last two configurations (the drawing ones) and removing the related code from the example, unless somebody needs those. Rendering performance seem a little outside of what the stress test is meant for, and it might be a better fit for a separate rendering/batching stress test.

[alice-i-cecile]

Edit: I'm thinking about removing the last two configurations (the drawing ones) and removing the related code from the example, unless somebody needs those. Rendering performance seem a little outside of what the stress test is meant for, and it might be a better fit for a separate rendering/batching stress test.

I agree with this.

[CptPotato]

Alright, the drawing related code is gone, I also switched to MinimalPlugins while I was at it.

I'm fairly happy with this as it is. Let me know if I should add/change anything!

[alice-i-cecile]

Looks good to me! I'm quite happy with this code, and it should be very useful for development.

Once I have another approval I'm happy to merge this in, and then I'd like to reshuffle the other stress tests into this folder.

[alice-i-cecile]

bors r+

[bors]

Pull request successfully merged into main.

Build succeeded:

• build-android
• build (nightly, ubuntu-latest)
• build (stable, macos-latest)
• build (stable, ubuntu-latest)
• build (stable, windows-latest)
• build-wasm (nightly, ubuntu-latest)
• build-wasm (stable, ubuntu-latest)
• check-benches
• check-doc
• check-markdown-links
• check-missing-examples-in-docs
• check-unused-dependencies
• ci
• markdownlint
• run-examples