GSoC 2024: Improvements to the Blender macOS User Interface Experience - Weekly Reports

[Brainzman]

Week 2

June 3 - 7

This week:

• Initial GHOST API for enabling/disabling per-window decorations
• Apply decorations based off the presence of global areas (topbar/statusbar)
• Researched and explored different ways of implementing custom titlebars decorations on macOS

Next Goals:

• Implement dynamic topbar padding for inline titlebar
• Custom Traffic Light (Close/Minimize/Maximize) button views for flexible placement
• Compute the dragging area, padding and traffic light placement based on the topbar layout rectangle
• Capture drag events directly within the main Blender content view, and use events to cancel drag on UI actions
• Extend the GHOST decoration API to accommodate these objectives

[Brainzman]

Week 3

June 10 - 14

This week:

• Independent window control button view for flexible inline, UI scale independent topbar decorations placement
  
  
• Early prototypes for non-inline window titlebars that follow the current Blender theme
  
  

Next Goals:

• Continue improving inline decorations with padding, proper dragging, etc… (see previous week’s goals)
• Tidy up with the aim of releasing the client-side window decorations as an experimental feature
• Start working on other macOS UX fix and features (Window Positioning bugs, iCloud integration, etc…)

[Brainzman]

Week 4

June 17 - 21

This Week:

Globally quite an unproductive week GSoC wise as I was busy with a personal project I couldn’t postpone from Thursday to Monday afternoon. However, starting tomorrow, I’ll have my entire week free to work on the project and catch back up.

Still, this week, I worked on non-inline titlebar macOS decoration for non-main windows (Preferences, popped-out editors, Render View, etc…) which I plan to release this week as a standalone PR along with a separate refactor PR of the Cocoa/Objective-C GHOST macOS layer.

Following that idea, I decided with my two mentors to postpone prototyping on the main inline window decoration task for now to focus on landing usable and finished macOS UI/UX deliverables for users to play with. And will thus next focus on fixing the macOS window sizing and placement bugs, and possibly work on globally improving window restoration, as was discussed in this PR.

Once that’s done, I will split my time between continuing work on inline window decorations (see previous reports), and working on other macOS UX deliverables, such as iCloud integration, improved text cursor behavior, better system interactions, maybe better app packaging using Universal Binaries, and if I have some additional time, some global UI papercuts improvements.

[sergey]

@Brainzman

maybe better app packaging using Universal Binaries

We do have a TODO about it, but it is aimed to be done as part of Steam packaging. Using universal binary for all other places has a downside of heavily increased download size. Which is especially unideal since the x86 macOS platforms seems to be on their way to be phased out.
I wouldn’t really prioritize this for the GSoC project.

[Brainzman]

@sergey

We do have a TODO about it, but it is aimed to be done as part of Steam packaging. Using universal binary for all other places has a downside of heavily increased download size. Which is especially unideal since the x86 macOS platforms seem to be on their way to be phased out.
I wouldn’t really prioritize this for the GSoC project.

Right yeah, I see what you mean, just to see, I did some quick tests to see what a universal binary build would cost us in terms of size. Here are the results on the latest commit from main (56fdfe5) using the default CMake Release config, with the universal binary created using lipo.

Binary	Size
arm64	156,4 MB
x86_64	170,9 MB
Universal	327,3 MB

This does make for an increase in binary size of 109%, effectively doubling it, as the two binaries are really just appended to each others, however, when we look at the total macOS app bundle size, we get:

.app Bundle	Size
arm64	812 MB
x86_64	866,8 MB
Universal	982,9 MB

Which gives us an increase of 21% from arm64 to Universal, which is still consequential, but has already way less of an impact on the total download size.

For me, the eventual rational for shipping Blender as an universal binary, like other FOSS softwares do, such as Godot, is less about guaranteeing multi-arch support, and more about making sure users don’t accidentally run Blender through Rosetta, thus lowering their performances, a problem I already ran into myself in the past after copying Blender builds between macs, and keeping old DMGs around. Especially since there’s no way of realizing that you’re actually running Blender through emulation, unless you inspect the application or use an external indicator like Silicon Info.

This situation might even actually degrade with time, as Intel Macs become more rare, Blender will still need to keep shipping x86_64 builds for as long as they’re officially supported, as such, the number of users potentially using the wrong builds and lowering their performances might actually increase with time. Although this can of course also be solved through clear Download guidelines, or eventually some sort of splash screen indicator that Blender is currently being run under Rosetta, using an Universal Binary might be a simpler solution.

On the technical side of things, CMake does most of the heavy lifting for us, to build an x86_64 binary on arm64, just setting the CMAKE_OSX_ARCHITECTURE=x86_64 build flag is enough, along with manually enabling the lib/macos_x64 submodule. Ideally, we’d want CMake to be able to perform the lipo concatenation itself using CMAKE_OSX_ARCHITECTURES=arm64;x86_64, which currently fails due to the library directory search not expecting multiple architectures, but I might dig a bit more into it later to see if that’s easily fixable.

As you said, this is certainly not an obvious tradeoff, and would certainly require more discussion with the Platform module, but all in all, it might be worth it as a way to globally make things simpler, guaranteeing good performances and simplifying release builds creation and distribution, especially considering that the end download size increase is not that consequential.

If you want to continue this discussion, feel free to either reply here or ping me on blender.chat if you prefer to not pollute this thread too much, I can also create another devtalk topic if you’d think that’s more appropriate.

[brecht]

@Brainzman

This does make for an increase in binary size of 109%, effectively doubling it, as the two binaries are really just appended to each others, however, when we look at the total macOS app bundle size, we get:

.app Bundle	Size
arm64	812 MB
x86_64	866,8 MB
Universal	982,9 MB

Did you consider the .dylib and .so (Python module) files? I would have expected those to add another 400MB - 500MB.

[Brainzman]

@brecht

Did you consider the .dylib and .so (Python module) files? I would have expected those to add another 400MB - 500MB.

Ah… Good point you’re right, indeed I didn’t factor in the rest of the executable libs that needs to be turned into universal binaries, as I just based myself off the existing arm64 bundle to quickly throw something together (which of course only worked on my machine, doing the same thing using the x86_64 bundle would cause a crash due to missing libs).

I took a stab at trying to properly generate Universal App Bundles for macOS. For this to work, all used libraries need to be concatenated using lipo just like the Blender executable itself. As a quick experiment, I wrote a simple Python script that creates the universal libraries by merging the lib/macos_arm64 and lib/macos_x64 directories into a lib/macos_universal directory, using lipo if the file is an executable or static/dynamic lib, recreating relative symlinks, and simply copying every other files, which I then fed into CMake.

After disabling SDL due to a weird Objective-C relate linker error I didn’t want to completely investigate, and running into a few linker warnings (which seems to be a known issue that’s fixed when using XCode as a generator) the result is a pretty staggering… 1.52GB Universal app bundle

Updated Table

.app Bundle	Size
arm64	812 MB
x86_64	866,8 MB
Universal	1.52 GB

As the resulting bundle basically doubles the download size, the whole idea of using these for Blender releases kinds of fall through. Still, I’ll probably turn the script and slight CMake modifications I made into a proof of concept PR for anyone who wants to experiment with this or want to be able to build a Universal Binary version of Blender for themselves (and eventually also as a base solution for the TODO PR @sergey mentioned). As a future side-task, I might also experiment with displaying a splash screen warning if Blender is being run through Rosetta to still address the problem I talked about above.

[sergey]

The way Steam delivery currently works is that it downloads builds from buildbot, repackages, and pushes them to steam. It does not do compilation. If we can have a script to which you can give two builds of Blender (x86 and arm64) that’d make it easy to attach it to the current pipeline. Do you think it is possible (it actually sounds you have similar script already)?

[Brainzman]

Ah right I see, yeah for sure that should be possible. The script I made merges two pre-compiled library directories together so that they can be used to compile a Universal Binary, but I should be able to adapt it for it to take two x86_64 and arm64 Blender app bundles instead for this to be done post-compilation.

It even somewhat simplifies things as we don’t have to deal with library symlinks and merging static libraries together to please the compiler, just merging the Blender executable and embedded dynamic libraries inside the bundle should be enough. I’ll modify the script in that sense, would you like me to drop it here once it’s done or do you prefer me opening a PR somewhere?

[sergey]

I think you can create a PR against the blender-devops. Maybe place it under blender-devops/buildbot/worker/blender. I forgot how exactly the Steam re-packaging work, but it seems it already does unpacking of DMG. Perhaps this could simplify some work for you. The details of that part are in the blender-devops/buildbot/worker/deploy/steam.py. Ideally we download both DMGs and call a function from your script.

Unless you want to dig into more details of this devops pipeline, you can provide script which you think will lets us do an universal build, and then we do the devops pipeline side of the changes.

[Brainzman]

Week 5

June 24 - 28

This Week:

• Finished implementing colored macOS Titlebar Client-Side window decorations that follow the Blender theme
  

  
  
  
  
  
  See the Pull Request for other screenshots, videos and details

• Objective-C Refactor Research
  The original plan for this week was to publish an initial GHOST Cocoa/Objective-C Refactor PR to serve as a clean base for the above macOS client-side decorations PR. However, as I started to work on the refactor, the scope of the task grew beyond what I anticipated for a simple PR, as the GHOST and overall Blender Objective-C code is largely outdated, not only in terms of style, but also in terms of API usage and memory management (we currently use the “old” Obj-C MRR memory management style, but newer code was written with the expectation of having automatic reference counting enabled, causing leaks among other major inconsistencies).
  I thus decided to postpone this refactor, in favor of publishing the above PR, as to then be able to focus on it in one block, going beyond simple stylistic changes (see Next Goals below) with the goal of providing Blender with an updated, modern and stable Objective-C software layer.

• Additional Researches on macOS Universal Binary builds (see previous messages in this thread)

Next Goals:

• Objective-C/Cocoa Refactor - In multiple steps/PRs:
  • 1st: Stylistic Refactor - Using @autoreleasepool instead of NSAutoReleasePool, includes/imports and comment cleaning, enforcing dot notation for setting properties (using foo.thing = c instead of [foo setThing:c]), etc…
  • 2nd: Memory Management - Enabling and switching to Automatic Reference Counting (ARC) and rewriting functions in that sense, fixing leaks and over-retains.
  • 3rd: Functional Refactor - Clean-up and switching to newer APIs, eventually leading to feature advancements such as multi-monitor window sizing and placement support.
• macOS Universal Binaries
  • Experiment with adding a splash screen notice when Blender is ran through Rosetta to warn against reduced performances
  • Implement Universal Binaries for macOS Steam packaging delivery
• Misc macOS/UI bugfixes, such as:
  • Fix floating window title not properly updating on editor changes
  • Fix eyedropper not working outside of the macOS Blender window anymore
• Following up on Colored Titlebar macOS decorations, continue the work on Inline Titlebar decorations (see previous reports)

[Brainzman]

Week 6

July 1 - 5

I ended up coming down with a pretty bad cold this week, which knocked me out for most of it. As a result, my work was mainly concentrated over the last few days. Here’s what I produced in this period:

This Week:

• Fixed a bug where floating window title would not properly update on editor change (Pull Request)

• Implemented a Splash Screen notice to warn against reduced performance if Blender is ran through Rosetta on macOS (Pull Request)
  

  

• Progress on the first phase of the Objective-C Refactor (see previous week’s goals), with a PR coming in the next few days

• Experimented with using AppKit’s NSColorSampler as an alternative eyedropper for picking color outside of the macOS Blender window, triggered when shift-clicking the eyedropper icon
  

  

• Additional Color Picker and UI related experiments:

  • Experimented with an improved layout for the Color Picker
  • Experimented with improved color field drag-and-drop throughout the Blender UI

Next goals:

See previous week’s goals, mainly focusing on Objective-C refactors from now on.

[Brainzman]

Week 7

July 8 - 12

Progress this last week has been quite steady, mainly focusing on the Objective-C refactor.

This Week:

• Continued the work on the first phase of the Objective-C refactor, the scope of which has broaden to now also cover the refactoring of macOS specific delegates and subclasses of AppKit objects
• Continued experiments on an improved Color Picker layout, with a WIP PR coming soon

Next Goals:

• Finish and publish the first phase of the Objective-C Refactor
• Continue on with the rest of previous goals (rest of the Objective-C refactor, Universal Binary Steam package delivery, Inline Titlebar Decorations, etc…). See the Week 5 Goals for more details

[Brainzman]

Week 8

July 15 - 19

Not that much overall progress this week, while I had originally hoped to release my work towards the end of the week, I ended up coming down with food poisoning which knocked me out for the better part of it.

This Week:

• Overall progress on main tasks (Objective-C Refactor and other experiments)
• Participated in Thursday’s UI Module Meeting
  • Showcased the project and discussed its direction
  • Got feedback on PRs and discussed their UI design
  • Discussed future macOS/UI related goals and targets (Cross-platform CSD, macOS image copy-pasting support, etc…)
  • See the Meeting Notes for more details

Next week:

Hoping that this sickness won’t last too long, my next goal is going to catch up on and release my current tasks, clear the accumulated backlog, and move on to new deliverables.

[Brainzman]

Week 9

July 22 - 26

Overall a much more productive week, here’s what has been accomplished in this period:

This Week:

• Objective-C Refactor
  • Finished the main Cocoa Refactor, overhauling style, memory management, AppKit subclasses, and more, making for a more solid and maintainable base for further macOS improvements. Final diff of +1594, -1671 changes
  • Ported the BLI_delete_soft function from Obj-C Runtime code to proper Objective-C
  • Objective-C allocations and release pool profiling, leaks corrected in the refactor
• New and Improved Color Picker Layout (PR Link)
  
  
• Experimented with new macOS specific deliverables taking advantages of the Cocoa Refactor (see Next Goals)

Next Goals:

• macOS Image Copy/Pasting Support
• macOS Titlebar filename and save indicator text improvements
• Add iCloud in the File Explorer sidebar and port the underlying macOS filesystem code to modern Objective-C
• More Objective-C Refactors (Carbon Deprecation, Metal code, etc…)

As a small bonus, macOS builds for the Colored macOS Titlebar PR are available here. Feel free to test it and share your feedback in this thread if you’re interested.

[Brainzman]

Week 10

July 29 - August 2

Apologies for the late report, ended up being quite busy towards the end of last week, which led me to essentially concentrate on polishing existing features and bug-fixing. However, the great news is that starting today, I’m officially on vacation, which will let me entirely focus on the project in the coming days, working on new deliverables and larger tasks.

This Week:

• Participated in Thursday’s UI Module Meeting
  • Got feedback on and discussed project deliverables/PRs
  • Showcased and discussed new Color related ideas (Color Picker layout rework, Color Datablock drag and drop, Improved Color Nodes)
  • Discussed future macOS integrations (Dark/Light Theme Auto-Switching, Finder Integration in the File browser, and more)
  • See the Meeting Notes
• Implemented requested changes and improvements on the Color Picker layout rework PR
• Decoration API Refactor and Implemented requested change on the Colored macOS Titlebar Decoration PR
• Fixed Interactive Docking support on High-DPI/Retina Displays and API clean-up

Next Week:

See last week’s goals, focusing on macOS deliverables and improvements, with some eventual more general UI deliverables.

[Brainzman]

Week 11

August 5 - 9

This Week (and start of the next):

• Participated in the last UI Meeting (13/05/2024)
  • Got further feedbacks on the project and its ongoing PRs, especially on the Color Picker Layout
  • Discussed macOS window placement and future plans to properly support multiple monitors
  • Quickly showcased the new Client-Side Decoration WM API, with a potential future implementation of Colored Titlebar Decoration on Windows 11 by Harley
  • General UI discussion, see the Meeting Notes for more details
• Continued Improvements on ongoing PRs
  • Color Picker layout rework PR
  • Colored macOS Titlebar Decoration PR
• Misc bug fixing PRs such as Fix: UI: Wrong single editor window titles due to unset area sub-spacetype #126224
• Finishing up the design of the new WM/GHOST Client-Side Decorations API deliverable as part of the Colored Titlebar PR.
  • Expandable flag-based API, featuring the possibility of applying multiple decorations style flags on a per-window basis
  • Provides Blender with a simple platform-independent and future-proof decoration API, allowing to create dynamic decorations using the current theme or window state
  • API overview/writeup coming soon once the design will be completely final
• macOS Image Copy/Pasting Support:

Demo - PR Coming Soon, some tweaks are still needed to properly cover all image source and destination types.

Next week:

As the very last week of GSoC (starting August 19th) will overlap with me starting a new (Blender related \o/) job, I’ll be less available in this final period. Next week will thus be focused on finishing and releasing in progress deliverables and refactors to start wrapping up the project.

[Brainzman]

Week 12

August 12 - 18

This Week

• Wrapped up the design of the new WM/GHOST Client-Side Decorations API, making the macOS Colored Titlebar PR ready for review. (See quick API overview)
  

  
  • Bonus: Windows 11 Colored Titlebar Decoration Port by @Harley (many thanks!)
    
    
    
    
    

• UI Bug fixes

  • #126379 - WIP Fix: Interactive Docking modal operator re-run glitch
  • #126383 - Fix: UI: Editor window(s) title update when using interactive docking
  • #126386 - Fix #126246: Missing Outliner redraw notifier when making datablock single user

• Released the macOS Image Clipboard Copy/Paste Support PR

  • Still marked as WIP for now as some design decision needs to be taken regarding support for image file references in clipboard (and not just image data), as well as tweaks to support full arbitrary bitmap format/encoding

Next week

Last week of GSoC! I will be mainly focusing on wrapping up everything and writing my final report.

[Brainzman]

Week 13

August 19 - 23

This Week

• Released the GHOST Objective-C Refactor set of PRs (Main Tracking Issue #126772)
  • Remove unsound use of const in Objective-C code #126768
  • Use idiomatic Obj-C objects for Window/App delegates #126767
  • Autoreleasepool and property dot-notation refactor #126771
  • Port BLI_delete_soft from objc_* runtime calls to proper Obj-C #126766
  • WindowViewCocoa refactor #126769
  • General Code Style cleanups #126770
• Published the project Final Report

[Brainzman]

Final Report - Improvements to the Blender macOS User Interface Experience - GSoC 2024

Summary

Over this summer, I got to work on improving the Blender macOS User Interface Experience. While the initial goal of the project was to add native macOS menubar support and inline titlebar decoration to Blender, the end goal ended up shifting towards more general macOS and general UI research, while still working on client-side window decorations, as outlined in this kickoff meeting note by my mentor Julian.

As such, this project ended up focusing on a new cross-platform API for client-side window decorations, combined with a practical colored titlebar macOS decoration implementation. I also worked on general macOS user experience and interface improvements, as well as backend refactors, while also experimenting with additional general UI enhancements.

Links and Process

• Original Proposal
• Feedback Thread
• Weekly Report Thread

During the project, I had regular weekly sync-up meetings with my mentor Julian, and for the first half of the project, my second co-mentor Farsthary. Towards the second half of the project, I also attended the bi-weekly UI module meetings, the notes of which you can find linked in the Weekly Reports.

Deliverables and Code

Colored Titlebar Decoration (macOS)

The main deliverable of this project, with probably the most impact on the Blender macOS end-user experience, are the Colored Titlebar Window Decorations. Providing native menu bar tinting to the Blender Window based on the current theme, blending the Blender window content with its titlebar, and improving its integration in the surrounding macOS environment.

Colored titlebar decoration before/after GIF, using the default Blender Theme and the macOS Light Theme to showcase the difference

Full macOS Titlebar Window Blending, featuring the Nordic Gold Theme

More Informations, Screenshots, and Implementation Details can be found in the Pull Request: macOS: Colored Titlebar Client-Side Decorations - GSoC 2024

Client-Side Decorations Window Manager API (General UI)

Another main deliverable of this GSoC project, and a byproduct of the Colored Titlebar Window Decoration implementation, is the creation of an expandable, flag-based, Window Manager Client-Side Decoration API. With the ability to create dynamic decorations based off the current Blender theme or window state, while also being as simple and platform-independent as possible. See the technical API overview for more details.

A great example of the use of this API, is the Proof-of-Concept implementation of the Colored Titlebar decoration on Windows 11 by Harley Acheson (@Harleya - many thanks!), using a minimum amount of Windows specific backend code, and sharing the same decoration logic as its macOS counterpart

See Harley’s PR comment for more details.

Objective-C Refactor (macOS)

To lay a solid foundation for future Blender macOS features, and modernize the (now more than 10 years old!) GHOST Cocoa backend, one of the main deliverables of this GSoC project is a global refactor of the Blender Objective-C code, which also serves as a proposal for a unified Objective-C code style. To help with code review, the final result was split into 5 Pull Requests, for a grand diff total of (+1781 -1887) lines (with some overlap between PRs).

• Main Tracking Issue #126772
  • Remove unsound use of const in Objective-C code #126768
  • Use idiomatic Obj-C objects for Window/App delegates #126767
  • Autoreleasepool and property dot-notation refactor #126771
  • Port BLI_delete_soft from objc_* runtime calls to proper Obj-C #126766
  • WindowViewCocoa refactor #126769
  • General Code Style cleanups #126770

Color Picker Layout Rework (General UI)

Note: Work In Progress GIF — Final design is still under review and subject to change.

Pull Request: UI: Color Picker layout rework #125675

Image Copy / Pasting Support (macOS)

Pull Request: macOS: Add Image Clipboard Copy/Paste Support #126574

Splash Screen notice to warn against reduced performance on Rosetta (macOS)

Note: Work In Progress — Further layout experiments and discussions are needed, with the general idea being approved.

Pull Request: macOS: Implement a Splash Screen notice to warn against reduced performance on Rosetta - GSoC 2024 #124382

Bugfixes

• Window Titles
  • Fix: Window title not properly updating on editor change #124304
  • Fix: UI: Wrong single editor window titles due to unset area sub-spacetype #126224
  • Fix: UI: Editor window(s) title update when using interactive docking #126383
• Interactive Docking (See Feature Info Thread)
  • Fix: Interactive Docking support on High-DPI/Retina Displays #125926
  • WIP Fix: Interactive Docking modal operator re-run glitch #126379
• General UI
  • Fix #126246: Missing Outliner redraw notifier when making datablock single user #126386
  • WM API: Improve naming of WM_window_pixels_* functions #125994
• Small documentation nitpicks
  • Fix bpy module typo #104880
  • Fix: RNA: Wrong doc description for WM_cursor_modal_set() / restore() #126227

Future Plans

Wrap up on-going work

Finish the code review of ongoing PRs, such as the UI: Color Picker layout rework, the macOS Rosetta Splash Screen Warning, etc…

Port old legacy client-side decorations Inline Titlebar experiment to the newly developed decoration API.

Legacy Inline Titlebar decoration experiment

Continue working on on-going experiments

• macOS Specific: Native Color Picker, Theme Auto-Switching, iCloud support, Full Multi-Monitor Window Placement Support, etc…
• General UI: Better main window titlebar, remove extra edges around main window
• Objective-C Refactor: Use modern Cocoa APIs, removing remaining legacy code, switching to ARC
• Color Related: New RGB node layout, Color Data-Block Drag-and-Drop in Node Tree, new Color Picker slider and wheel shapes
• Improvements to macOS Steam Deployment via Universal Binaries
• And more!

Thanks!

I would like to give a very warm thanks to everyone who supported me throughout this project, and most importantly to my mentor Julian Eisel, for his valuable guidance and feedbacks, and for keeping me motivated through regular sync-up meetings. As well as to my second co-mentor, Farsthary for his time and assistance during the first half of this project.

I would also like to thanks the Blender UI Module team, Pablo Vazquez for his interest in my works and his feedbacks, and especially Harley Acheson, for taking the time to answer my UI related questions, providing tons of useful feedbacks, and reviewing my code.

Finally I want to express my overall gratitudes to all the Blender Developers, and most importantly, the Blender Community. Looking forward to continue contributing to Blender and its beautiful user interface in the future!