Developer.blender.org - Call for comments and participation

This is actually something which I think is an advantage. Pull requests only work if the person doing the pull request has his git repository shared online. Which would mean that either developer.blender.org needs to host git repositories for everybody, or people need to publish their repositories themselves with all the hassle that brings.

I do dislike that arc diff squashes all commits into one, though.

I think GitLab would be the least risky choice in terms of health of the project, because there are many other closed and open source projects relying on it and there is real funding behind it. If GitLab the company goes under I think it’s likely to continue in some other way.

The alternatives seem to be mostly driven by volunteers and there are few public instances, seemingly none used by a big open source project. Gitea development for example still happens on GitHub, which is understandable but shows the maturity of the project. I think such projects are much more likely to become unmaintained, though I hope they don’t.

In terms of features I think it’s mixed, I think GitLab is clearly better for us as-is, but also would be harder to improve to suit our needs.

2 Likes

If I understand the situation gitea is in, then it is not driven by volunteers but they have 3-4 full-time developers working on it.

These are paid via a fund like we do with Blender.
Most of the funding seems to come from small to medium sized companies. Last time I looked the majority of sponsors seems to be located in China.

There is a publicly viewable internal vote every year were they decide who should get funds to work full-time on the project. There seems to be more willing developers than there is funding so IIRC, they also have a bounty system where people can chip in for funding developers outside of the core team to finish a task.

hosted on github and not self-hosted

There is a ticket about this on their github page.
The gist of it is that they have been able to self-host with all the features they need for a long time. They have even done so will all their repositories besides the main “gitea” repo. (They have around 10 other repositories besides the main one.)

The two major reasons for this seems to be two fold:

  1. The cost of self hosting the main repo would be quite large. They could do it, but why spend money and admin power on that instead of hiring more core developers? Most of their clients don’t care if the development happens on a self-hosted instance. They only care if the product itself is good enough.

  2. Self hosting will make the project more isolated as it won’t then be part of the github developer ecosystem. Perhaps not a big downside. But it would mean potentially less community engagement both in the issue tracker and pull request wise.

To me, even if gitea doesn’t have the same kind of funding that gitlab has, I think it is a safer bet because it is fully open source. (And there seems to be less “vendor lock in” than with gitlab because it goes all in on supporting 3rd party CI systems etc)

I have had experiences in the past of going with half open solutions. Those where bought up by Oracle and the licensing got restrictive as heck while the free tiers were swiftly eliminated and prices jacked up. Migration was very painful and costly.

I don’t want to see blender getting into such an uncomfortable situation.

11 Likes

From Gitea - Open Collective I understood the funding was small, but if there are 3-4 fulltime developers (presumably from other sources) that’s more reassuring.

1 Like

Unfortunately this would mean the Gitea funding could be affected significantly by increased western sanctions, which seem rather likely looking into the future. In general the Chinese economy is hard to predict. OTOH if a big player like Blender switches to Gitea that may help diversify their funding.

I think it’s very hard to predict the longer term health of the choices we have, so doesn’t seem like something we can give much weight in our decision making. Worst case is we have to do another transition in a few years, but since the platforms would use the more common github-like workflow, that might be less painful than porting from Phabricator. There are plenty of other things that may go wrong that could lead to that scenario.

Note that I could be remembering wrong, so if it is important, we should double check so that it is correct.

This is what I remember from when I did some initial research last year, just to be clear.

I think that 2. also applies to Blender.

There is an argument to be made for moving to GitHub.com, as

  • from all of these options, it is the most likely to stay operational
  • it already has significant buy in from the wider open source and FOSS community
  • it would not require Blender to host user forks or allow any contributor to branch - either of which would be necessary to allow for the Pull Request based workflow that Github, Gitlab and Gitea are build around

IMO this should be seriously discussed and not discarded for ideological reasons. Even if GitHub is deemed too proprietary, I think it would be worth it to evaluate what openness in this context here means and to what extent it can be achieved / what parts of it are worthwhile pursuing.
Does it matter what software runs on the blender.org servers?

2 Likes

Unless @Ton radically changed his views on this recently, i don’t see github being an option.

5 Likes

ok, i guess if we take that as gospel, that also outrules gitlab premium edition.

maybe I’m mistaken, but from quick googling i don’t think either gitlab or gitea allow you to upload patches. So how will the development workflow in the future look like?

i guess there would have to be some access control with one role that would allow any user to create a branch, but only be able to make commits on own branches.
and another role for trusted users who are able to make commits on any branch.
I know that in GitLab you can protect certain branches, but I don’t know if there is a way to whitelist the branches a user is allowed to commit on.

alternative to access control would be to allow anyone to fork - but does BF want to store all of the user forks? Maybe pull requests from other hosted copies of the same git repo can be made - but that would just mean that users fork on github and then make the pull request on gitlab / gitea (if that even is possible)

to be honest i dont know what you can setup with build in tools in gitlab (or gitea) and what kind of stuff would need custom stuff. but i think this is the aspect that requires most attention rn, because afaik the self hosted versions of gitea and gitlab are designed to be run within an organisation, where you can have a base level of trust against users.

Initially there should be some clearing about the self-hosting option.

  • Self hosting: If strategically, is for the best interest of Blender Foundation, and also with extensive customization, it would meet the needs of the core developers in a better way.
  • Thirdparty service: If Blender Foundation, has no problem at all either, with self hosting either service-based hosting. It means that service-based hosting wins because it eliminates in-house maintenance.

In other way of thinking, it can go like this having a decision matrix and you attach points (backed with some fact). You can play with this diagram and expand as needed, do your own experiment and see if it works.


       | business  |            |  important | advanced |       | 
       | stability | popularity |  features  | features | other | score
       |---------------------------------------------------------
gitea  |    ?      |     ?      |            |          |       | 
github |    10     |     10     |            |          |       | 20... 
gitlab |    ?      |     ?      |            |          |       |  
other  |    ?      |     ?      |            |          |       | 


A little more info about the criteria:

  • business stability: most important is that if the company has completed a the full cycle. As for example why github is a [10] because has completed all of the business phases (startup/growth/acquisition) and now it has settled for lifetime, as long as it remains popular it will remain as it is without surprises. While in contrast other companies, are experiencing growth-antagonism phase right now, they have future ahead of them, they could get a [8] or [9] etc.
  • popularity: not to say it intuitively, but looking at the cold data is even more clear. https://trends.google.com/trends/explore?geo=GR&q=github,gitlab,gitea but the point is not to use a marketing strategy exactly - we are not selling donuts, rather than consider if it actually popularity plays a role whatsoever, how popularity translates better to the specific needs of Blender?
  • important features: these are must-have git features that core Blender developers need. In that regard everything gets a [10]. However this brings another aspect, of whether or not self-hosting gets a [10] while other service-hosting drop points.
  • advanced features: same as before, but these are nice-to-have features however they make a clear difference and cover some very specific needs that Blender developers know they want. Also if these features are solved with self hosting it means that you create a new entry vertically (self-hosted-gitlab) and set it to a [10].
  • other: use this to expand with many more aspects if needed
1 Like

There aren’t that many options as far as I can see. To my surprise there is no “truly” open solution which appears very future proof. The point Ton raises is certainly important. It is still worthwhile to weight it against the cons of the other options.
He made the point when Phabricator was the number one option for Blender. With that one gone, the whole balance changes quite a bit.

Contrary to the good points about how widespread it is, GitHub is prone to political decisions.

Let’s assume a hypothetical situation where one of the allies of the USA, countryA is in a conflict or a war with another nation, countryB. The US government orders GitHub to block all the users and the activity from countryB. This does not sound very unlikely nowadays. And by law Github followed such orders in the past.

We know that Blender has a lot of contributors from all over the world. Self-hosting the development backend and the frontend is the only neutral solution in these times.

On the other hand the new backend could allow logins with Github and Gitlab accounts, that might be a good compromise.

8 Likes

I don’t really get what that would accomplish? It would not really make pulling from github/gitlab easier I think?

It would make so that people with those accounts would not need to create individual Blender accounts to contribute on developer.blender.org. It is just a convenience thing.

2 Likes

I would like to add bundled add-on development to the discussion. Has it been considered at all when looking for a new platform? Should it remain as one repository with everything lumped in together, potentially with one main “issue” for each add-on to keep track of things, or should each add-on have a separate repo for its development?

5 Likes

Just wanted to drop a note here that I’ve been following the discussions going on here and am deeply thankful for the amount of insightful views that people are providing on all manner of aspects. Sorry for having been quiet about that so far.

A big part of what I’m doing is collecting and collating different concerns and considerations that people have about what is good/bad/ugly about the options that we have available.

I want to take a moment to address a recurring theme in the discussion above, however, which touches on the subject of licences and openness.

It’s no secret that Blender has a proud history and culture revolving being Open Source. This is true for Blender and its related tools and projects as much as is possible. This is of course not accidental.

For this reason, preference goes out to seeking solutions that are fully open source if those can be made to fit the needs of the development community. This is not an ‘at all costs’ approach, however, but should be seen as a strong internal compass.

For that reason, for example, running on a full Github setup is an unlikely way forward. But neither is it impossible if other routes turn out to be fruitless. The process right now is geared towards evaluating what’s out there in an honest manner. The comments, reports, suggestions and concerns above are helping in this a lot.

Cheers,

19 Likes

yo, i have an idea, idk if its possible but what if you use discourse as the frontend and throw in some bot accounts to interface with a git backend?

Since no one has commented on this yet, I’m going to guess that it hasn’t been discussed very much yet and y’all want to hear my thoughts on it :wink:

Blender’s bundled add-on development is a bit of a mess currently IMO, add-ons generally don’t have a central landing page where you can see what’s happening with them, issues and diffs are all lumped in together, if the development isn’t being conducted on some external repository somewhere, and developers with commit access can change bundled add-on code without the author’s knowledge or consent (this isn’t to say that contributions aren’t welcome, but it can cause problems like ⚓ T95962 Sapling Addon now broken because of "SPDX-License-Identifier" header).

Since it seems that Blender will be adopting the fork/pull request paradigm, my current thinking is that I would like to see each bundled add-on get it’s own repo to manage development (bugs, tasks, landing page, PRs, etc.), have scripts/buildbot incorporate the master branch from each of these into the daily builds/releases, and a central listing of bundled add-ons so that the repos are easily findable. Or perhaps bundled add-on development should move entirely to external repos on GitHub, GitLab, Codeberg, etc., but then there is the challenge of synchronizing them with the rest of Blender.

Now I know this doesn’t directly help with picking the new development platform, but I think this is a good time to discuss it and to keep add-on development in mind when looking at the different platforms in case one platform is better suited for something like this than another.

Also, how bundled add-on development proceeds will be influenced by the new policies that have been made recently, and that are hopefully being discussed and clarified here Add-ons Policy Clarification/Discussion so feel free to chime in there if anyone has any thoughts on that as well.

4 Likes

Some critical issues have not yet been raised, which can all be encapsulated in the form of the GNU Ethical Repository Criteria. There are also some evaluations by these criteria, although the list of evaluated sites and software is far from complete.

Blender is a flagship free software program and should be a shining example of freedom. Relying on proprietary software, no matter how much the proprietors claim to be friendly to “open source” or want to give a discount for their proprietary software, is a complete betrayal of this goal of leading by example. No matter whether the entire platform has critical freedom issues (and privacy issues for that matter), eg. Github, or whether the software is only partially free, eg. GitLab, anything less than 100% freedom is a joke. Both of these sites currently score an F (unusable) in the GNU ERC, the worst possible grade.

Should the Blender project self-host or use an existing platform? Ideally the former, as this allows the Blender project to control its own infrastructure, software, and computers. The latter is okay, but not ideal, as it would mean the Blender project would lose a lot of the aforementioned control.

In any case, contributing without an account, anonymously over Tor, and without running JS, especially not proprietary JS, are an absolute bare minimum, which as I mentioned previously, is all encapsulated in the GNU ERC.

I vow to delete this account once this public discussion is concluded, and I resent having to make it in the first place in order to participate in this discussion. If I refused to make a temporary account, who knows whether anybody would have raised the issues I have, and as a result the final decision may have been disasterous.

Forges which allow communication by email are a solution, which leads me to lean towards self-hosting SourceHut as a preferred option, although it has a few issues which need fixing first, namely that it recommends non-libre OSs in its CI features and that it has not had an official “stable” release yet, although it’s already rock-solid.

2 Likes

wasn’t Sourcehut the one that got its grade lowered by the FSF because it didn’t call linux “GNU/Linux”?