GSoC 2020: Regression Testing Framework - Feedback

Contributing to Blender Summer of Code
1

Hey everyone, this thread is for suggesting improvements, discussion on the regression testing framework and any discussions related to that, as the long(edited: now short) title suggests :D.

Here is the link to my Proposal.
And here is the link to Weekly reports, to keep an eye if I am working properly or not :wink:

5 Likes
2

Really excited to see where this goes! Some folks see testing as something that slows development down, but in reality, a good test framework will speed things up. It allows devs to be fearless if and when they want to make changes, at any time in the cycle. Getting to the “good test framework” part is pretty difficult though.

A few comments and suggestions:

  • The Hook modifier should be testable without user interaction. An object hooked to an empty should get a deformation going. So that can be tweaked in your spreadsheet I think.
  • If possible, when there’s a set of changes that are immediately useful to have in master, I think it would be ok to submit those as soon as they are ready rather than waiting until September+ for your entire gsoc branch to be merged. This is also advantageous for review meaning things will go faster later on.
  • Keep the spirit of patches like D7590 alive. In other words, if new tests completely replace older ones, please let’s clean up the older ones :slight_smile: It reduces confusion and mess.

Good luck!

3 Likes
3

Here is some of the work I will be doing in the coming week and I would like some feedback on the following:

  • including support for creating a Vertex Group in the script
  • open discussion to generating “Helper” objects from the script (whether they will useful or not)

An example of how the prototype works-

createTestObjects({‘Cube’: ‘Cube1’, ‘Cone’:‘MyCone’})

One has to enter the Object Type and Object Name and the object is created, so after the above script is run, the Blender file will have 2 objects “Cube1” and “MyCone”.
It will work on the already existing blend file if present otherwise create a new one.

Feel free to provide your insights on whether this is a good approach and any other suggestions.

Here is the link to the survey again.

1 Like
4

Hey!
I would like some urgent feedback on which tests are failing on other systems and whether it is an environment-specific issue or is it random. Steps to help me out through this process is-

  • Checkout my branch soc-2020-testing-frameworks
  • Download the supporting blend files from D8507 and place them in the correct folders as mentioned in the above link.
  • Is there any specific test I am interested in? Yes, the Cloth tests.
  • How should I run the tests? Please have a look here
  • If its too long for you, follow these
    • Go to the build folder of the blender compiled from my branch.
    • Run ctest -R cloth -C Release -VV (here -R is for regex and we are choosing the cloth test, and -C is the configuration)

Thanks!

1 Like
5

Currently ClothPressure and ClothSelfCollision tests are failing, I am commenting it out for the time being.