Scrubbing timeline slow when "Blender File" is open in Outliner

I have a file with a bunch of heavy files linked into a single master file.
It runs with good performance (ca. 25 FPS) in the viewport if I hit the play button.
Performance crawls to a halt if I scrub the timeline. Perhaps 1 FPS.

The thing is, this only happens when I have Outliner set to “Blender File”. If I set the Outliner to anything else, scrubbing works perfectly fine.
It appears that this menu being open makes the linked files update or redraw on each scrubbed frame or something like that.

Is that expected behavior or should I file a bug report?

It happens in Blender 2.93 but I also tested in 3.0 alpha and 2.92 and it is the same there.