Thank you so much for the tip! I updated the launch.json (when starting debugger from sidebar) and settings.json (when starting debugger through CMake tools) accordingly.
With add-ons development, you can just reload the add-on in Blender after modifications in order for them to take effect. I thought the “Live edit” feature in Blender’s text editor could to something similar with built-in scripts, but I had no such luck. I know picking up changes is even tricky in a regular Python REPL after having already imported a module. If anybody can think of a way to reload the built-in scripts without having to close Blender, please let me know!
I also hope this discussion will solve the Python auto-completion with bpy.