Discussion of D12622: Fallback Font Stack

I’m hoping to use this post as a means to invite larger discussion of this idea from core developers, with the hope of including D12622 early in the 3.3 release schedule.

Current Fonts

Our interface currently outputs text characters with FreeType using our two “meta-fonts”. These fonts are just huge fonts that we have created ourselves, starting with a copy of “DejaVu” and then adding from “DroidSans” as we improved our language coverage over the years. You can find these two files in the datafiles/fonts folder as “droidsans.ttf” (a variable spaced font of 5.2MB) and “bmonofont-i18n.ttf” (a monospaced font of 5.4MB).

These fonts contain about 54,000 characters, covering most of the world’s largest languages. But adding any new characters is becoming progressively harder. Many language fonts will define a character as being an overlay on top of another, but that source character can differ slightly between languages. This is not a problem when using separate fonts, but causes issues when they are smushed together.

The maximum number of characters in any single TrueType or OpenType font is 65,535 so we are already getting close to that maximum. Obviously there is no limit of characters we have access to if we use separate font files.

There are also license considerations when creating our own fonts. We have to trust that anyone adding new glyphs is doing so from a font that is suitably licensed and we aren’t going to go through each stroke ensuring that is done correctly.

For users a primary downside with our current fonts is that we allow them to use any font they wish, but it then replaces ours. So selecting the ideal font for you will mean that you will get greatly-reduced character coverage. In fact there are many language-specific fonts that contain only that language’s alphabet and nothing else, not even the regular Latin characters (A-Z, etc).

Proposed Solution

As described in D12622 I am wanting to deconstruct our meta-fonts into separate font files. And in doing so solve our current problems and add new features and possibilities.

The root of that datafiles/fonts folder would instead contain two plain, regular, updated versions of “DejaVu”, one variable spaced (DejaVuSans.ttf at 740 KB), the other monospaced (DejaVuSansMono.ttf at 333 KB). For the Latin characters these fonts are identical to what we have now, so users won’t notice any change. Although eventually I’d like to swap these two out for something more modern (and that are Variable).

These two fonts only contain Latin and some Greek, which is why they are so much smaller. For all the other characters we add a new “fallback” folder that would contain any number of other font files. In my patch I propose a collection of fonts that cover about 1.5 billion more people than we do now, plus math symbols, other symbols, an icon font, and a “last resort” font.

All the fonts in the “fallback” folder are loaded blindly, so you can just add a new font there without needing to make any other changes. And “last resort” fonts (that use format 13 cmaps) are detected automatically.

Fallback Strategy

As you can guess, the idea is that if a character is not found in the main fonts we then look in the fallback fonts, with “last resort” being the last one.

But I don’t want to literally go from font to font to font. Each font contains a set of “coverage bits”: a set of four 32-bit flags indicating that it has substantial coverage of a Unicode block. So if we need a character we just look up which block it belongs to and then go straight to a font that contains it, or use the “last resort” font, or “not.def” if not found anywhere. This character-to-unicode-block lookup will also be needed when we do bidirectional reordering and for complex text shaping (like with Arabic).

This means we don’t have to touch files unnecessarily. With FreeType caching D13137: BLF WIP: FreeType Caching we can have fonts without FreeType faces loaded. This would mean that if you never use a particular font, like Javanese, it would not have to be kept in memory at all. So no penalty of having good coverage and the quickest access to the characters you actually use.

Monospaced Characters

One unusual thing I am proposing here is to only have a single stack of fallback fonts, not two. So our usages of monospaced characters (primarily in text editor) would use a base monospaced font for Latin characters but would then use the variable spaced fonts for fallback. This patch includes such conversion and it works very well. It adjusts width, stroke weight, and alignment, and even handles wcwidth (double-width monospaced) and symbol fonts:

For comparison, this the same document using current master, so only using our monospaced metafont. Obviously there are some coverage holes, but if you compare the non-Latin that is common between them (Armenian, Chinese, Georgian, Japanese, Korean) you will find little difference.

Variable Fonts

You might notice that almost all the fonts in my proposed list of fallback fonts are “variable” versions. These kinds of fonts can vary in weight and width and this will come in handy when we are supporting this font type.

Any thoughts or concerns?

13 Likes

@Harleya does this mean we have additional icons that we can ise as part of the N-panel tab names and Workspace tabs?
Could you perhaps add the default Blender Icons to the font, so we could use those for workspaces & n-panel tabs as well? This could help sort one of the long standing issues for users who would like to have icons instead of names. Lots of addons nowadays support renaming their tab in the N-panel, so getting the icons in there as a glyph would then be an added bonus. :wink:

E: The only thing to keep in mind is that for the N-panel, there has to be a set of icons that is rotate by 90 degrees so that they align properly with the rest of the UI.

Yes, adding an icon font this way would add no overhead yet allow the ability to show icons anywhere there is text. So yes, would be great as a pool for addons to use. They would rotate in the same way that text rotates though, but we could output them separately if need be.

The current icons might be able to one day be added to a separate font, because the OpenType SVG format supports all the features it needs (filled regions of varying opacity), we are currently lacking convenient ways to edit them. Hopefully Font Forge will add that one of these days.

3 Likes

I don’t know which tools you would require from Font Forge, so could you maybe file a feature request over there then? I think a lot of people would be happy if we could make the icons happen this way :grin: