Hello! My name is Eveline Anderson and I am very excited to start helping out on the Flamenco project as a “newbie” to the open-source contributing world. I love Blender and was elated at the opportunity to contribute to the community. This post will be intended to hold my weekly reports, and my proposal.
The following is an outline of my proposal, but my full proposal can be found with this link: Eveline’s Flamenco Improvement Proposal
Synopsis
Around a year ago, Flamenco—a rendering-farm Blender addon—was released to improve the workflow for animation artists across the globe. It is extremely successful, but still has many key functionalities that can greatly improve it. The goal of this project is to build upon the foundation of Flamenco, such as designing a web interface for the manager configuration, introducing RNA overrides, and improving stability, to bring it to its full potential.
Benefits
As mentioned previously, the idea of Flamenco is perfect for all animation artists, as it speeds up the rendering workflow (which is something that is desperately needed) in a simple, yet functional way. Building upon the base that Flamenco has already set up will improve the following factors:
Ease of Use — To make an application more popular, users must be enticed to use it. Therefore, the more user-friendly interfaces are implemented, the better the odds are that they will want to use it.
Removal of Redundant Tasks – A common word that tends to be used in a negative light is the verb “grinding” which is another word for repetitive, redundant actions. The goal is to remove as many redundant tasks from Flamenco as possible. This can be accomplished by removing jobs and tasks that constantly need to be resubmitted instead of just using a function that can set certain properties to make the process more automated.
Improved Stability/Predictability – An application could have the most amazing features, but it would be of no use to a client if it is not stable. While Flamenco is NOT an unstable application, the improvements in areas such as the Worker can help eliminate any unexpected crashes or outcomes that could frustrate a user.
Deliverables
The Flamenco project is already a well-built application, but the deliverables of the project are listed below are to improve the groundwork:
Create a web interface for Flamenco Manager so that it is possible to configure the manager from its web interface.
Design job types for various tasks, such as distributed rendering of single images, and more powerful control over parameters for studios.
Introduce per-Worker logs on the Manager, for introspection and debugging.
Introduce RNA Overrides.
Secondary Tasks – Improve the stability and predictability of the jobs.
User Documentation and Notes.
Flamenco Development Branch
The following link is to the specific branch I will be pushing code to. If you’d like to see my progress, please check out the link. Eveline’s Forked Flamenco Branch
Once GSoC has begun, I will post my weekly reports in the comments. Thank you so much again for the opportunity and I’m happy to be here!
During this period, I took some time to learn and familiarize myself with Golang and unit testing. Because I knew JavaScript, it wasn’t all that difficult to pick up on the syntax. However, unit tests are a completely different story and took a much greater time to truly understand.
My mentor (Dr. Sybren Stüvel) and I also devised a plan on how to best to tackle the Google Summer of Code proposals. Flamenco is a far bigger project than I am used to working on, so I needed to get familiar with the Flamenco project’s structure and code. We decided to start with smaller fixes before diving into the heavier parts.
Accomplishments
As mentioned before, we decided to cover some changes that would not be as costly with time, due to my lack of familiarity with Golang. This week, I–with the help of my mentor–was able to accomplish the following:
Understand Go/Unit testing to a basic level
Discussed with mentor the structure and get a broad overview of the Flamenco project
Create and submit a PR request for #99549 When sending Workers offline, remember their previous status
Discovered a new issue to fix #104219 Unit tests are timezone-dependent
Struggles / Blockers
I think the largest struggle was being in a project that I knew little about (coding-wise). I love using Blender and Flamenco, but never really went into the deep coding structure of the program. It was easy to feel overwhelmed, so I’m grateful that my mentor patiently guided me through the design of Flamenco.
A basic struggle, but knowing very little Golang and unit testing made it difficult to understand some parts. Thankfully, the fix that I needed to work on was very small. But, it still took me a great deal of time to actually solve the issue, which can be frustrating.
I now truly understand a developer’s struggle in fixing one issue and causing 20 more. The unit test that I created, broke both the TestWorkerSignoffStatusChangeRequest test and the TestWorkerSignoffTaskRequeue test. It was terrifying to see those errors, and took more time to solve them than the actual bug.
One small issue that my mentor and I discovered is that some of the unit tests were dependent on the timezone. Since I was in a completely different timezone than my mentor (Europe vs America), it took some time to figure out why I was getting some errors and my mentor wasn’t.
Lessons Learned
Staying organized. I thought I was very organized when it came to coding projects. But, being actually put IN a huge one has taught me how important it is to follow the steps of first understanding, secondly assessing, and THEN coming up with a game plan.
Start small, then grow bigger. I used to be the student who would take the harder classes first, then do the easier ones. It worked in school, but in a big collaborative assignment, it’s far better for me to start small to get used to my surroundings, and then move onto bigger hurdles.
Keep up with communication. I really thank my mentor for being so open and willing in communicating with me. Stuff that took me hours to solve, ended up being solved in a matter of minutes thanks to his help.
Notes
Documentation. I might end up writing some documentation on the problems I solved and explain what the solution does. It might be helpful for some in the future, but the documentation might be but on the official documentation page anyways.
Change of plans. The biggest goal right now is to get Flamenco 3.3 up and running for release. Though some of my items in my proposal will help, it’s not completely focused on Flamenco 3.3. Therefore, as of now, I have been assigned and am focusing on the following tasks to help speed up the release:
1. Rename Worker Clusters to tags.
2. Web interface for CRUD operations of worker clusters.
3. Documentation for the Worker Clusters.
Any new changes will be updated in my proposal. But, as always, anyone can see the changes I’ve made in the link to the git branch that I forked.
Side Note: I was today years old when I learned there was such a thing as a git GUI. My mentor introduced me to it and it’s a game changer for managing git commands/changes. I recommended anyone to try it out!
This week was busy, but less stressful than last week. I learned a lot more about Flamenco’s code architecture. There was a request to rename clusters to tags, this mean having to go through the code and change the instances. Thankfully, I didn’t have to do much coding, but this was still a long task.
Renaming of clusters to tasks throughout Flamenco
Discussing the web interface for worker clusters/tasks
Struggles / Blockers
I still have some things to work on with the renaming. I thought it would be a shorter task until I searched and there were over 2,000 instances of renaming that I have to do in 67 different files. I was then notified that there was an easier way to do it through an OpenAPI, so I made a new branch and reverted back to my previous changes.
I’m not very talented in designing web interfaces. I’m far better at implementing them. So, I’ll need to reach out more to the community to discuss the interface design options (if you have an idea, please feel free to me in blender.chat!). This might take some time.
Lessons Learned
I spent way too long manually going through the Flamenco architecture. As my mentor said, if I notice the manual labor is getting too much, it’s time to make it automated! I’ll definitely be more mindful of that next time.
This week was a bit more challenging than last week. It took some time to come up with a plan of attack for the interface creation, as well as getting used to communication as a community. With the help of my mentor, we pushed through and was able to come up with a good mockup for the interface!
Communicated with community regarding worker clusters interface
Designed a figma mockup for the interface
Communicated with mentor regarding the rough-draft of the design
Continue to rename the worker clusters to their proper name (tag)
Struggles / Blockers
Worker tags. Since the renaming of worker clusters to worker tags, I’ve still got a ways to go. Some files are not very happy with the rename and cause errors that aren’t easy to trace. I’m going through each file, trying to tackle the “root” files and then work my way down. That way, there’s a less likely chance for major errors to come up that I cannot read.
Multitasking. I’m not the most amazing multitasker, but having to focus on restructuring variables while worrying about the creation of a new interface is challenging because if any errors come up from the variables, it can cause my interface to not even show up. So, I’m working on balancing between the renaming and interface development.
Communication. Sometimes, it’s a bit difficult to get questions answered from the community, and it’s even more difficult to start when there’s not a whole lot of feedback. I also am learning to ask the right questions, so it’s a lesson for me to learn as well.
Notes
The path I’m going on is a bit different than my original proposal. I am working on the tasks to get Flamenco 3.3 up and running, so the detour affecting my schedule just a bit. I am going to be rewriting my proposal in a later time to reflect the changes that I’ve made, but any changes (again) can be seen in my forked repo.
This week is fun and I’m really starting to get into the “fun” stuff with Flamenco. I cannot wait to see what’s next!
I learned a lot more regarding the git gui this week, which was really nice. I learned a better workflow in handling git commits/merges/squashes/etc. through my mentor, as well as good practices for committing. It was a lot of fun to learn how to make git a little easier. Some other accomplishments this week:
Created Pull Request for Worker Tags for clustering #104204 regarding the cluster renaming. Just one step closer to getting Flamenco v3.3 released!
Implementing the worker tag interface.
Struggles / Blockers
My mentor and I decided that I committed way too many times for one pull request. So, he helped me to unstage the commits and do it in a more organized manner. Then, we found a small bug in the code. Thankfully, he was able to find out what caused it!
The default formatting in my VSCode can really mess up the original formatting of the code structure, which can be frustrating. Git GUI really helps outs with this, as my mentor showed, so that I don’t push code that makes the formatting weird.
Getting the code changes of the interface to actually show up on the website is trickier than I thought. Vue is not a framework of JavaScript I use often, but I’m sure with a little catch up, it won’t be too bad.
Notes
The more I am shown the power of the git GUI, the more I really love it. Even after Google Summer of Code, this is something that I will definitely continue to keep using. I’m also happy to see that the Flamenco channel is becoming more active! It’s really nice to see everyone work on it
Wow, it’s already been 5 weeks and we’re half way through! This week was mostly about bug fixing and version upgrading. There was an animation convention this week, so it was definitely a busier time for my mentor! Thankfully, he was still able to help me solve a majority of problems. Some things I accomplished this week:
Issue fixing for pull request: Worker Tags for clustering #104204
Updating of programming libraries
Struggles / Blockers
I had to learn the hard way that downloading libraries through the Ubuntu machine terminal is not always reliable. A great deal of Flamenco’s code was not working because of my outdated libraries. If I would have known this in the beginning, I wouldn’t have spent time trying to figure out errors and why certain errors kept showing up despite no solutions being found. I’m glad it was a simple fix!
This week, there was a couple of issues that needed to be addressed on my remote project, but thanks to the help of some other developers, I was able to resolve them and hopefully finish off the main task of clusters and tags!
Resolved CORS issue in Flamenco for testing in a dev environment (thanks to @michael-2 and @rwtnitin on Blender chat!)
Fixed Javascript issue for Worker Tags for clustering #104204. Hopefully, this is the last issue with this ticket!
Assigned new task, Web interface: fetch version on SocketIO reconnect (and maybe reload) #99410
Struggles / Blockers
Git. Git has been the biggest joy and the biggest pain to me, though I have no doubt it’s just because I haven’t mastered it properly. With simple things like pushing, pulling, branching, I survived through it. But when it comes to more complex issues (merging, reverting old changes, solving git errors), it became more difficult to get through. Thankfully, my mentor helped me through, but this took us over an hour to resolve.
CORS. I had a great deal of difficulty solving the CORS issue, so I couldn’t properly test any changes I made. At first I tried by myself to fix it, but no amount of Stack Overflow, Googling, or documentation reading was solving my problem. The community members helped me to solve it, so I was really glad for the help.
Notes
Branching is becoming more and more useful the more I work on this project. My mentor advised me to think of branching as a bookmark for when I want to make certain changes. This made it very easy for me to visualize, so I definitely want to start using branching more to my advantage!
This week was truly all about git and I’m so glad to get the exposure to using it more in-depth!
First mid-term has passed. Many thanks to my mentor for all his help and guidance . But, it’s not over yet, and I still have a ways to go.
Accomplishments:
For this week, it was relatively slow with the starting the new task and trying to complete the previous worker clusters task.
Issue Worker Tags for clustering #104204 is another step closer to being finished and Flamenco version 3.3 being released!
Created PR for Web interface: fetch version on SocketIO reconnect (and maybe reload) #99410. I hope for it to be fully finished this week.
Struggles / Blockers
The biggest blocker I faced was trying to figure out where to start with issue #99410 as there seemed to be some parts that I was not familiar with. However, thanks to the well written documentation for Flamenco, I found a starting point and was able to start the first PR! It’s nice to have the help from the community as well.
This week was incredibly busy due to some of the vagaries of my personal matters. Nonetheless, I am appreciative to all my mentor’s support and help.
Completed PR #99410 for SocketIO reconnect
Begin working on PR #104204 Web interface for CRUD operations of worker tags.
Struggle / Blockers
Git. It is an incredibly useful tool, but in the wrong hands (my hands) it is incredibly destructive. I had to spend a good day and a half to undo, redo, and revert in my branch. It was very frustrating to deal with. I’m thankful to my mentor and the community for their advice. The sheer tediousness of git definitely ate up some time when I could have been finishing the PR.
SocketIO. It was a completely new concept to me. Though flamenco already had some documentation regarding the usage of SocketIO, it took me a moment to wrap my head around its functions to complete to PR task.
Personal matters. Some personal matters led to my overall packed schedule. My mentor was very patient, but it would have been nice if these matters didn’t arrive in the first place. It would have been easier to finish early.
Notes
My mentor has pointed out the downfalls of using bash, and I’m starting to see them as well. It’s really handy to be able to customize the terminal, and I think I need to start playing around with it more.
This is a very small report as minimal items were accomplished during week 9.
Accomplishments
As I was just trying to catch up from last week, I am still progressing in working on the PR: #104204 Web interface for CRUD operations of worker tags. I’m hoping to get more done this week as this set me behind a bit. My mentor will be out a good chunk of next week, so I will be relying on the community’s help! I’m very excited
GSoC is coming to an end, sadly! I hope to be able to extend and help with releasing Flamenco v3.3! The following are some of my accomplishments for the week:
Create table functions for interface page of the worker tags
Add ability to fetch, create, and delete tags
Struggles/Blockers
VueJS isn’t extremely difficult to master, but it is rough to understand in the beginning and some of the deeper level items took me a few minutes to really comprehend.
Grabbing the tags from the right place. I needed to make sure I was actually grabbing and constantly watching the tags from the actual manager. Same with deleting, creating, and updating. So, that will take some extra time.
Tabulator. I’ve never even heard of it before, but I love it’s design format for the table. It’s definitely not as simple to make as I thought, but I’m trying to stick it through with the documentation provided.
Notes
My goal for next week is to get a PR for the first version of the interface. Though, it may not get reviewed soon, I hope to at least have this done by next week so that the development of the tags will be one step closer to being finished.
To my mentor who is going to a big conference, I wish you all the best on your presentation, I know you’ll do well
This week was quite productive! Despite some blockers, I was able to work around some issues so that I could make a little progress.
Implemented more CRUD (delete and update) options for the interface grabbed from the backend.
Reviewed documentation for Tabulator
Struggle/Blockers
XHR errors. I have no clue what those errors mean. The strangest thing is that the errors that show up on my end don’t show up on my mentor’s end. It’s difficult to figure something like that out.
Notes
I was hoping to submit my PR by now, but at the very least, I’m glad that I was still able to make some progress. Hopefully I’ll be able to push something by the end of week 12. That way the workers tags for Flamenco will at least be slightly polished. I’m so happy with my experience here, it has been so much fun!
This will be one of my last weeks with GSoC!! I’m sad that it’s coming to an end, but I am so excited to complete one of the biggest tasks I’ve done since I’ve been here, the web interface for worker tags.
Accomplishments
After my mentor helped me with the XHR error I was receiving, I was able to get a lot more accomplished and testing done.
Created PR #104244 for the Web Interface
Implemented Tabulator into the interface
Rewrote creation and deletion of tags to be compatible with Tabulator
Edited some styling to match the other
Struggles/Blockers
Updating Items. There is still some difficulty for me to do this. There’s a great deal of documentation on Tabulator on how to manually update the table, but what I’m looking for is a bit different from what I’m reading. So, it might have to take some more out of the box thinking.
CSS Styling. It’s not very easy to style Tabulator and add classes to the table that’s being generated based on the tags. I want it to look exactly like the other tabs (jobs, workers), but I don’t want to do it manually, rather I wish to use the classes in CSS that have already been made.
Notes
I just have a few more tweaks and I am very close. I know I can work on this after GSoC ends as well, but it would be such a cherry ontop to finish before final evaluations start!
The finale is here ! I was able to finish up some of the last major features that were really needed for the interface. There’s still a couple more things that I would like to touch up with it before I call it “finished”, but I got all the “hard” parts finished.
Accomplishments
My mentor was on vacation, but still gave me a lot of help during the time that I needed it. I was able to accomplishment the following:
Implemented editing feature in Tabulator
Names and descriptions edited now shown in the table, show up in the API
Edited styling for the table and window side
Added default description option
Work in Progress
I still have to connect the table to SocketIO, but I’ll communicate this more when my mentor comes back. This shouldn’t take too long and hopefully the PR will land next week. I’ll probably update this final post as more information comes, but that’s all for now!
. But, it’s not over yet, and I still have a ways to go.
.
