Just a quick announcement that we are changing the submodule configuration of the main blender.git repository. This will be done for both blender-v3.5-release and main branches.
The very terse version is that you’ll need to run make update and it will automatically take care of converting the submodules to the new structure.
The fuller version can be read in the design proposal which includes motivational part and the actual design of the changed status. The important parts:
release/scripts are moved to the scripts
addons and addons_contrib are simple Git checkouts, which are ignored as per .gitignore
source/tools are moved to the tools and are part of the main repository. The repository used for submodule will be archived.
release/datafiles/locale is moved to locale and is part of the main repository. The repository used for submodule will be archived as well.
Another aspect of the change is that the make update will support Github style of remote organization (origin remote pointing to thy fork, upstream remote pointing to the upstream blender/blender.git).
EDIT 1: Just to clarify: the addons and addons_contrib are still separate from blender.git repositories (blender-addons.git and blender-addons-contrib.git). They are just checked out using slightly different git commands (regular git clone instead of git submodule).
EDIT 2: The new version of the make update is needed to bring the repository to the proper state. So, git pull (or other means of updating your working branch) is needed to grab the new version of the make update script.
I ran make update a few times to make sure it’s the latest version. Already tried deleting the build folder and compiling from scratch. Seems to be related to this change. Do you know what could be causing the errors?
Don’t see why this commit triggered the behavior, but it seems like it’s unrelated wrongness that causes it
msgfmt has a TBB dependency though bf_blenlib. In the release config the MSVC linker is smart enough to figure out it is not used, and discards the dependency hence you can run msgfmt without needing tbb.dll available, debug however leaves the dependency intact and requires some runtime dll’s.
Now if you have an existing build folder, tbb_debug.dll will already be sitting next to msgfmt.exe and all will be well, a clean folder however the runtime dll’s only get copied during the install phase, long long after msgfmt is supposed to run. causing msgfmt to error out since its runtime requirements are not met.
I have a fix, will try to get that in later today.
Added an edit to state that a new version of make update script is needed.
Would be nice if the script somehow did the initial pull itself, to grab the new version of self and re-run. But that’s for another day.
Going to a revision prior to this change brings back the old make update which had those issues for bisecting:
The main blender.git can not be updated because it is detached head. And you do not want to update the HEAD anyway.
The decision about what libraries and tests to pull did not work reliably either because the tag/branch of SVN repositories and submodules was based on the main repository branch name.
If you go back in history, run make update it will re-initialize the submodules, and most likely to pull the latest submodule state. This will be really the latest state for addons and addons_contrib. For translations and tools that’d be the state of repositories prior to this change.
If you go back to the latest main branch and run make update one of the things will happen:
If you never had the addons and addons_contrib initialized then the script will move the submodules over to the new location (so that if there is any local changes to them they are not lost)
If you had the addons and addons_contrib initialized you’ll see unstaged addons and addons_contrib folders under their old locations, and the make update will update the check-outs at the new locations.
Starting from this change and onwards, using make update in the middle of bisect will allow you to update libraries to the proper state since the code is now based on the BKE_blender_version.h.
I suggest that you jus try the bisect, see if there is anything wrong with it, and come with a proposal or workaround how to get try better.
I’ve always had the libraries (and tests) in a git-svn clone for bisecting, to prevent hammering the svn server all the time. And also because svn is rather sluggish on my (mobile) internet and re-updating the complete libs folder over svn for every bisect step when you are flipping back and forth over a libs change was excruciatingly slow.
I saw some noise on the chat about git svn in make update, but can’t really find anything in make_update.py (admittedly I didn’t look too hard, just did some searches).
Now I know there are plans to switch the libs over to git lfs sometime so probably it’s no use trying to advocate the addition of a git-svn path in make_update.py, but I just wanted to throw it out here.
Since the switch to new libs in a different folder from a different repository my setup broke somewhat (still need to do manual stuff when crossing the lib switch date) but for me having the libs in git-svn made a night&day difference when bisecting because it never needed to re-download.
Can ‘make update’ (and build process) please recognize following tree as valid (if they exists) and, not re-download/pull these (addons/(-contrib)) repositories?
First of all, thank you for this reorganization.
The request is primarily because, with current(new) re-pulling addons into scripts make it harder (read messy) to keep archive of the these repositories as addons and addons-contrib does not actually exists within git tree (submodule or otherwise).
I use ‘git pull --all --rebase’ and previously it was ‘git pull --all --rebase && git submodule update --init --recursive --remote --rebase’.
I keep repositories with upper structure to keep all these repositories properly tracked.
So now, when ‘make update’ is called addons and addons-contrib gets re-pulled into scripts, which makes it duplicate efforts. And, these two actually does not exists within blender repository, as they are git ignored. Maybe think of it like how ‘lib/*’ are treated during build.
You should be able to use make source_archive. This is what the release builds are using.
I do not think it is realistically to fit everyone’s workflow. Keeping the addons and addons-contrib inside of the scripts was an explicit choice to:
Support the workflows when developers don’t use make install and rely on symlinking to speedup iteration process.
Keep the Python API refactor doable form inside of a text editor without need to go into complicated setup with multi-work-tree workspaces. It is still possible to do modifications to RNA and do global search in all scripts.
So it is like a trade-offs no matter which path you take. Hopefully, the current situation is here for a not so long time, and eventually community addons will be moved to the extensions platform, and the core addons will become part of the main repository.