Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

First setup of github action for automated blender testing #584

Merged
merged 44 commits into from
Jun 8, 2024

Conversation

TheDuckCow
Copy link
Member

This PR aims to add automated testing for new PRs by downloading blender and running our test suite.

Get ready for a large number of one-line commits as I slowly work on getting this to work. Not actually ready to review, I'm just working on the triggers primarily.

@TheDuckCow
Copy link
Member Author

Making progress, but seems at least one run had a crash. Still not 100% all files are in place, but the fact that it was trying to load the file seems to be a positive sign. Going to do some more isolated tests for just single sets of test classes at a time.

...
Read blend: "/home/runner/work/MCprep/MCprep/MCprep_addon/MCprep_resources/rigs/hostile/drowned - HissingCreeper-thefunnypie2.blend"
Warning: region type 4 missing in space type "Info" (id: 7) - removing region
Read blend: "/home/runner/work/MCprep/MCprep/MCprep_addon/MCprep_resources/rigs/hostile/PolarBear - PixelFrosty.blend"
Warning: region type 4 missing in space type "Info" (id: 7) - removing region
Read blend: "/home/runner/work/MCprep/MCprep/MCprep_addon/MCprep_resources/rigs/hostile/nether hostile - Boxscape.blend"
Warning: region type 4 missing in space type "Info" (id: 7) - removing region
Warning: region type 5 missing in space type "Buttons" (id: 4) - removing region
Writing: /tmp/PolarBear - PixelFrosty.crash.txt

Removing:  /home/runner/work/MCprep/MCprep/MCprep_addon/mcprep_addon_tracker.json
Removing:  /home/runner/work/MCprep/MCprep/mcprep_addon_trackerid.json
-------------------------------------------------------------------------------
bversion   	ran_tests	ran	skips	failed	errors
-------------------------------------------------------------------------------
tests took 2s to run, ending with code 1
Error: Process completed with exit code 1.

@TheDuckCow
Copy link
Member Author

I'm going to wager "no" to files not all being in place, given the latest run limited to just the materials tests:

-------------------------------------------------------------------------------
bversion   	ran_tests	ran	skips	failed	errors
-------------------------------------------------------------------------------
(4.1.1)   	MaterialsTest	24	0	17	

@TheDuckCow
Copy link
Member Author

Ok I've ruled it down to this: the actual addon install wasn't happening as far as I could tell, sorted now by ensuring we explicitly create the target install folder before running bpy addon build.

With that working now, more of the tests are passing, but there's a few others still. Going one class at a time. Might have a json / env loading issue, since we're not picking up this file (likely affecting the tests in question). I'll get back to this tomororw.

  File "/home/runner/work/MCprep/MCprep/MCprep_addon/util.py", line 64, in apply_noncolor_data
    options = env.json_data["non_color_options"]
              ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: 'non_color_options'

@TheDuckCow
Copy link
Member Author

TheDuckCow commented Jun 8, 2024

Making progress at understanding the area that went wrong. It does seem it's loading the mcprep data json, but somehow we only have these keys:

Path existing? /home/runner/work/MCprep/MCprep/MCprep_addon/MCprep_resources/mcprep_data.json  >  True
JSON data loaded, here's the contents
 --------------
['blocks', 'make_real', 'mob_skip_prep']

...

  File "/home/runner/work/MCprep/MCprep/MCprep_addon/util.py", line 64, in apply_noncolor_data
    options = env.json_data["non_color_options"]
              ~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^
KeyError: 'non_color_options'

... it's dropping/not loading the "non_color_options" which these tests depend on. I can't figure out where this partial json is coming from though, seems very strange.

@TheDuckCow
Copy link
Member Author

Oh. The problem is that we are not using the checked in json file, but rather extracting the one from MCprep_addon_v3.4.3.zip... and somehow that release indeed doesn't have the noncolor data in the json (and for some reason was using a really old release). Working on the fix.

@TheDuckCow
Copy link
Member Author

TheDuckCow commented Jun 8, 2024

Welp, that's official! Tests are running 100% on github actions across all (at least for blender 4.1.1). I'll do a separate test in a moment across all the targeted blender versions. This run's results:

https:/Moo-Ack-Productions/MCprep/actions/runs/9431750753/job/25980734109?pr=584#step:10:13

bversion,ran_tests,ran,skips,failed,errors
(4. 1. 1),all_tests,65,2,0,No errors

@TheDuckCow
Copy link
Member Author

Hey @StandingPadAnimations this is finally ready for review. Looks like it runs pretty stable now too, which is great. Once this is merged into main (after merging into dev first, ofc) we will then also have an option to go to the actions tab and select which versions of blender to run against per the choice enum in the workflow title. By default to save on github minutes, I'm just having it run the 4.1.1 build. But the matrix array should work to run against all those other versions, useful for release prep.

Let me know if you have any feedback. As a side note, other than making sure the right files like the json and ahem after git lfs pull got put into place, the tests did run fine. I still think it's very helpful for you to be able to run the tests locally, so maybe there's a bit more to look into. We could huddle sometime to sort it out.

@TheDuckCow
Copy link
Member Author

As a complete note to self, something rather bizzare I just noticed: the github minutes are counting against my own account as opposed to the Moo-Ack! Productions org quota. Doesn't matter since A) they are the same and B) we'd have to go real crazy to use up all minutes, as in like 2,000 individual commits-crazy (onto branches already opened as PRs). So I'm not concerned about that at this stage. Worst case scenario, we can reduce the tests to only run on manual dispatch and on PR opened (vs every commit, as it is now).

Copy link
Collaborator

@StandingPadAnimations StandingPadAnimations left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me

@TheDuckCow TheDuckCow merged commit 9aeeaba into dev Jun 8, 2024
1 check passed
@TheDuckCow TheDuckCow deleted the 575-automatically-run-mcprep-tests branch June 8, 2024 22:18
@TheDuckCow TheDuckCow linked an issue Jun 9, 2024 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Automatically run MCprep tests
2 participants