2020-04-23 - Development Infrastructure Plan of Attack

Blender Development Meetings
#1

Present: Brecht Van Lommel, Dalai Felinto, Sergey Sharybin

The goal was to determine what is the status of the current plans, and how to take the workload from Nathan Letwory given that he will focus mostly on release coordination for the upcoming month(s).

Topics:

  • Docker image for Phabricator
  • Canned responses for bug triaging
  • Automated deployment
  • Continuous integration
  • Crashpad

Docker image for Phabricator

Actions: Nathan to send current progress as patch review.

  • The current project is not yet finished.
  • Assess the situation afterwards.

Canned responses for bug triaging

Actions: Dalai and Sergey will tackle this after 2.83 is out.

Automated deployment

Actions: Nathan to handle documentation and automation.

  • Before tackling automation, documentation should come first.
  • Once automation is working then buildbot integration can be considered.
  • Status of platforms:
    • Steam: Not automated yet.
    • Snap: Needs update on script, already automated.
    • MS store: Not automated yet.
    • Any other platform (rpm, bpy) should need a more involved discussion including corresponding platform maintainers and chief architect.

Continuous integration

Actions: Need to hire a development support → write a job call, Sergey and Dalai.

  • Initial plan: clang-format
  • Auto-compiling and testing every commit on git.
    • Separate builder
    • Strip code sign and upload.
  • Add compiler and tests for patches
  • If easier first let developer hand-pick a patch to build.

Crashpad

Actions: None.

  • If there is a good backtrace without server components it would be better.
  • Not in the agenda.
#2

Please elaborate.

#3

For example, a way to make “release with info symbols” builds available for the users.

#4

For Crashpad the point was that if there is a simple way to get better backtraces on Windows without having to maintain server infrastructure, that would already help.

Given that Nathan will not be working on development support in the next few months and we don’t have another developer dedicated to it yet, setting up the crashpad server infrastructure is not something we’d considered critical enough to ask e.g. Sergey or another core developer to do.

1 Like
#5

Seems fair enough, from the description it sounded like having a stracktrace on the client would be somehow better than having an automated, low effort (for the user) reporting infrastructure, which would be a little weird.

I don’t see getting a stack-trace on the client being a technical problem, what has kept us sofar was the 100+ megs it adds for the .pdb file (no way to shrink those)

If we’re OK with that i’ll make it happen

1 Like
#6

Before we go yay/nay on this, this is the expected file sizes that we’d be dealing with

Full PDB (has functionname + source file + line number information)

blender.pdb           476,385,280
blender.pdb.zip       107,589,619
blender.pdb.ultra.7z   51,739,074

Stripped PDB (function name only)

blender_stripped.pdb     70,635,520
blender_stripped.pdb.zip 17,520,765 
blender_stripped.pdb.7z   8,175,591

Examples

Full:

19: BLI_system_backtrace - K:\BlenderGit\blender\source\blender\blenlib\intern\system.c:121
18: windows_exception_handler - K:\BlenderGit\blender\source\creator\creator_signals.c:282
17: UnhandledExceptionFilter - 0x7FF9705AFD00
16: memset - 0x7FF9734A2F80
15: _C_specific_handler - 0x7FF97348C5C0
14: _chkstk - 0x7FF9734A10B0
13: RtlRaiseException - 0x7FF973469E70
12: KiUserExceptionDispatcher - 0x7FF97349FE10
11: wm_debug_menu_exec - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm_operators.c:1633
10: wm_operator_exec - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm_event_system.c:989
9: ui_apply_but_funcs_after - K:\BlenderGit\blender\source\blender\editors\interface\interface_handlers.c:911
8: ui_popup_handler - K:\BlenderGit\blender\source\blender\editors\interface\interface_handlers.c:10843
7: wm_handlers_do_intern - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm_event_system.c:2725
6: wm_handlers_do - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm_event_system.c:2839
5: wm_event_do_handlers - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm_event_system.c:3234
4: WM_main - K:\BlenderGit\blender\source\blender\windowmanager\intern\wm.c:453
3: main - K:\BlenderGit\blender\source\creator\creator.c:526
2: __scrt_common_main_seh - d:\agent\_work\1\s\src\vctools\crt\vcstartup\src\startup\exe_common.inl:288
1: BaseThreadInitThunk - 0x7FF972FD7BC0
0: RtlUserThreadStart - 0x7FF97346CE30

Stripped

19: BLI_system_backtrace - 0x7FF6CBF38700
18: windows_exception_handler - 0x7FF6C989BB00
17: UnhandledExceptionFilter - 0x7FF9705AFD00
16: memset - 0x7FF9734A2F80
15: _C_specific_handler - 0x7FF97348C5C0
14: _chkstk - 0x7FF9734A10B0
13: RtlRaiseException - 0x7FF973469E70
12: KiUserExceptionDispatcher - 0x7FF97349FE10
11: wm_debug_menu_exec - 0x7FF6C9B46E90
10: wm_operator_exec - 0x7FF6C9B3FA90
9: ui_apply_but_funcs_after - 0x7FF6CA0CC670
8: ui_popup_handler - 0x7FF6CA0DA6F0
7: wm_handlers_do_intern - 0x7FF6C9B3EB70
6: wm_handlers_do - 0x7FF6C9B3E1C0
5: wm_event_do_handlers - 0x7FF6C9B3B9D0
4: WM_main - 0x7FF6C9B272D0
3: main - 0x7FF6C9897950
2: __scrt_common_main_seh - 0x7FF6CC488DE8
1: BaseThreadInitThunk - 0x7FF972FD7BC0
0: RtlUserThreadStart - 0x7FF97346CE30```

Offering the debug symbols as an optional download is possible, but that would require resources from the infra team.

#7

Including that stripped PDB in daily builds seems reasonable to me.

1 Like
#8

Work in progress in D7520 .

1 Like