GSoC 2020: Info Editor Improvements - Weekly Reports

[grzelins]

I will be posting my weekly progress here in this thread. I make notes and designs on my wiki page.
My branch is called soc-2020-info-editor, it can be viewed online

May (in general):

• searching thru code and woking on design Project Details

[grzelins]

Week 1: June 1 - 7

• reading more docs about architecture
• I am still working on Project Details , still reading through code
• changes to keymap and UI (zebra background, selection) in info editor, experiments already on my branch soc-2020-info-editor, details in commit messages (in future I will document changes in phabricator, I am still learning though) D7950

Next Week’s goals:

• create design task based on Project Details
• get more familiar with dev process on phabricator
• create UI for filtering reports by type (wrok started in D6926)
• expose debug flags in UI

[grzelins]

Week 2: June 8 - 14

• technical: exploring idea: can we make single backend for text, info and console editor? They are all text based fields. Possible benefits:
  – syntax highlighting in info editor
  – wrap text in info editor
  – autocomplete popup in console the same as in text editor
  – in future: vertical scroling will be implemented once for every view (work started in D4054)
• getting familiar with UI C api (for exposing debug flags in UI)
• experimenting with search functionality
  

[julianeisel]

@grzelins

technical: exploring idea: can we make single backend for text, info and console editor? They are all text based fields.

In principle I agree, there should be a way to share logic. However I see this as a much bigger project than what can be handled as part of this GSoC project.

We have multiple use-cases for text editing: Text editor, console, text widget (hopefully multi-line soon), 3D text object edit mode, the VSE team was recently looking into a way to edit text strips within the preview region, etc. For all these a decent experience is needed, with all the expected cursor navigation (arrow keys, modifier + arrow keys, home key, etc), select-navigation, shortcuts (e.g. Ctrl+A to select all), copy/paste, … And in some cases you’d want syntax highlighting, line-wrapping and I think some basic rich text editing features and spell checking are also something we might want soon.

Point is, yes we should unify this somehow. But I think this should be done “the right way” from the get-go, with a careful design that allows all these things. My idea is to have a reusable text editing widget at the core that manages all or most of this, with options and/or variants to extend it further.

My suggestion is that you keep the focus on your project. It’s easy to get side tracked by things like this, so that you end up delivering too little.

[grzelins]

Thank for summary.
That was also my conclusion, it is too big for this project. I was mainly looking if there is (easy) way to get syntax highlighting in info editor.

[Harleya]

The Info Editor uses some icons to represent different classes of reports. Many of those icons were just selected from what was already available by some amateur without much thought really. So if you want anything different, or better, or just something else, post a quick note on this thread and something new will magically appear for you.

[grzelins]

Week 3: June 15 - 22

Unfortunately I could not work on blender this week as I prioritized studies (this is final week of studies, exams). Due to pandemic this is very disorganised both from my and my teacher’s perspective.

[grzelins]

Week 4: June 23 - 28

• made easy to read design T78164 (with subtasks), anything what I do will be linked there
• working on implementing showing logs in info editor D8147
  – it is not on my branch yet, it crashes too often
  – getting familiar with list API, alocationg memory, also pointers gave me a headache this week
  – revise T78214 based on what I learned this week
  
  

[grzelins]

Week 5: June 29 - July 05

• working on D8147
  – it is not really stable right now, to even start, run --factory-startup
• updated my branch - it was stale for a while as D8147 was crashing too much to merge
• print to log conversion
  – if you care about commits 1 2 3 and more
  – reports that can not be diplayed (for example versioning reports) will be logged (they will be visible in console and info editor)
  – new loggers: wm.session, wm.job, blenloader.readfile, blenloader.undo, blenloader.versioning, bke.report
  – todo enable some loggers by default: blenloader.*, wm.session, blenloader.versioning (log format can be shortened)
• experimental support for showing python eval logs (stdin, stderr) as report:
  
  

And here is your eye candy:

[grzelins]

Week 6: July 06 - July 12

• working on reports of type: look in console for detail and gathering reports on blender startup
• investigating debug flag usage in blender and compiling Logging - best practices (feedback needed!)
• investigate how different log libraries work (like google glog, which is already used in cycles)
• investigate how to make a list of available loggers (probably system similar to makesdna?)
• separate use of verbosity and severity in logs:
  – previous: 4 severity levels (fatal, error, warn, info), now: 5 levels (+ verbose severity)
  – previous: severity info has levels, now: verbosity has levels, info is just info (no levels)
  – previously: --log-level controls info level, severity warn and error will always show if logger is on, now: --log-severity for severity, --log-verbosity for level of detail in severity verbose
  – if there is a need, I will add severity debug - only available in debug builds
  – although this might be considered being picky, I think it is important to use common conventions (as far as I know them), I am open to discussion
• converting print to logs:
  – G_DEBUG_WM has now almost now uses in code, completely replaced by logs
  – G_DEBUG_EVENTS, G_DEBUG_JOBS is completely replaced by logs
  – add logs to render statistics (log every 30-th status update)

[grzelins]

Week 7: July 13 - July 19

• investigate how to make a list of available loggers (continuation)
• converting print to logs
• looking for a way to expose in UI features from global structs
• investigating stability issues

[grzelins]

Week 8: July 20 - July 26

• experimenting with backend/drawing stuff
  – it seems that I can get syntax highlighting working in info editor
  – console also can get at least some syntax colors
  
  
• I spent much time trying to track down random crashes, but did not get anywhere

There is not much more to report. Getting through drawing code is challenging.

[ManuelGrad]

uiii … thats sexy!

[grzelins]

Week 9: July 27 - August 2

• working on exposing various log features in UI (preferences)
  – there are some (implementation) details to work out: should this settings be persistent? override setting with command line arguments
  
  

• filter report by type - minor tweaks in implementation and UI (experiments)
• convert clog to prints https://developer.blender.org/rBefb967e504e9fcc4572c452bed9fbd155cbebfed
  – add log severity debug - available only in debug builds
  – experimental: add to logging functions for measuting time. Rationable: every time there is time measurement, code is commented or guarded by some macro. Dead code is not good, I already found quite a few broken prints, no biggie, but we can do better it, ex. look at this:

#if 0
#  include "PIL_time_utildefines.h"
#else
#  define TIMEIT_BENCH(expr, id) (expr)
#endif

[grzelins]

Week 10: August 3 - August 9

This week I focused on completing UX design, I made technical layout of desired functionalities, you can find it here: GSoC 2020: Info Editor Improvements - Discussion and Suggestions (logs, reports & friends)
I encounter some complicated problems like:

• I want to save log/debug settings in preferences, but also override them from command line. This makes for non trivial logic
• obvious issue that I missed: before preferences are read, debug flags will not be applied
• command line debug flags sometimes trigger special logic, ex. --debug enables MEM_set_memory_debug(), what was completely missed in existing api bpy.app.debug and I also missed it. In some cases debug is checked only during init (like GHOST) and it does not matter what you do with this flag afterwards…

Also /blenkernel, /blenfont, /blenloader, /windowmanager is now completely converted to logs. Now i am going through /bmesh. I spend some time there to solve some more complicated log-cases.

[grzelins]

Week 11: August 10 - August 16

• daily build are back up, sorry, there were down for so long, I did not notice (I use Linux)
• mostly backend changes that made filtering work (backend is not done, I am working on multiline text display)
  – each filter has properties: inverse, case sensitive, use globbing (pattern matching)
  – operators for selecting logs and reports were separated
• fun comand to try changes: blender --debug --log-severity 0 --log-level -1 --log "wm.session,wm.operators,bke.report". You can later adjust them in preferences
• UI:
  – separate log filtering and formatting to different popovers
  – add separate menu items for log and reports

• add copy operators (remember you can use quick favorites in info editor):
  

• next week to come (maybe not all):
  – log operators: mute selected log types, function, …
  – polishing (debug features in user preferences, report drawing, various todo’s left in code …)
  – I will investigate autoscroll options
  – logs can change during selecting and you end up selecting sth different
  – auto refresh logs
  – search in logs uses only log message…
  – log syntax highlighting would be nice…
  – highlighting found text would be nice…

[grzelins]

Week 12: August 17 - August 23

• add mute operators to context menu:
  
• remove padding from multirow text:
  
• added autoscroll - autoscroll was actually the default behaviour, what makes it impossible to debug continuous log messages (like mouse movement, window manager bus), now it can be disabled
• fix selection synchronization: when selecting continuous log messages it was possible to for logs to update when selecting what resulted in shifted selection
• implement log delete operator
• now info editor will refresh on new log arrival
• I revisited patch for showing python output in info editor (fixed and improved)
• I was experimenting with python api for logging see this post
• started submiting patches for review

Next to come

• I noticed performance issues
• I have initial idea how out how to deal with third party (extern) libraries (specifically cycles and libmv, the one that use glog) and integrate them into Info Editor. I will do make design, but it will not make it into this gsoc. I will post later on discussion thread.

[grzelins]

Week 13: August 24 - August 30

• updated parent task my GSoC parent task T78164 with patches D8677, D8655, D8691 and task T80072