Has any one seen python module imports stop working in 2.82.
I have a script that imports a python file, addon_helper.py. The $PYTHONPATH is set to the the directory where the python file is. The import works in 2.81, but broke in 2.82 2 days ago. It was working in 2.82 until Friday.
I caught this using Travis, here is the first failing run:
https://travis-ci.org/douglaskastle/blender-fake-addon/builds/638638634
Has something changed?
I saw something the other day in the commit logs about clearing out PYTHONPATH because people were using it to bleep things up by importing the wrong numpy instead of Blender’s copy. I’ll see if I can find the commit.
Possible work-arounds off the top of my head:
Put the files where they belong in Blender’s directory structure.
Append to sys.path before you do the import.
Interesting, I am probably using python a bit weird here. I use it to add an addon for testing and then remove it when complete. They are effectively being used to put things in the correct blender directory structure. I have never has to mess with the sys.path for blender before, but if it works …
If you could find the commit that would be awesome!
there was a change there to prevent pythonpath wreaking havoc when populated with blender incompatible python verisons.
see https://developer.blender.org/D6598 for details.
Here you go. The important bit:
To use Python’s environment vars, pass the argument:
–python-use-system-env
Commit: 7c2f0074f3fe2411daa7a6e351d7cbc535246871
Author: Campbell Barton
Date: Thu Jan 16 21:11:05 2020 +1100
Branches: blender-v2.82-release
https://developer.blender.org/rB7c2f0074f3fe2411daa7a6e351d7cbc535246871
Python: disable environment variables by default
This avoids the problem where Blender doesn’t start because
the PYTHONPATH points to an incompatible Python version,
see T72807.
Previously we chose to assume people who set the PYTHONPATH know what
they’re doing, however users may have set this for non Blender projects.
So it’s not obvious that this is the cause of Blender not to launch
on their system.
To use Python’s environment vars, pass the argument:
–python-use-system-env
Note that this only impacts Python run-time environment variables
documented in python --help, Access from os.environ remains.
Ugh! Ok that’s it!
I was setting that value using one script, to set the PYTHONPATH to cwd + “/scripts”, and then another sub script was called that uses that directory to pull in its imports.
The problem with sys.path is that it is not communicative. Setting it in one script does not mean it will be set is another/sub script.
Possible solution is to set it in the toplevel file to a unique environment variable, like “LOCAL_PYTHONPATH” and then in the sub script explicitly append it to the sys.path.
Thanks for pointing me in the right direction and finding the issue to refer to.
It is a bit strange, as a regular python user I would expect access to PYTHONPATH by default.
That’s my intuition too, I resisted changing this since in general I think it’s good for Blender to have a “default” Python environment, it’s just that we kept getting reports of Blender failing to start (terrible user experience & time consuming for developers to troubleshoot). This way Python developers can still get a standard Python environment, they just need to pass an argument to Blender.
My inclination would be to add the flag to disable the PYTHONPATH and default is what it was. But i hear you on trying to clean up the user experience.
However i have also had much fun trying to get the python in blender playing well, or at least not interfering with system python.
I myself am now unblocked now that i now the behavior is a feature and not a bug.
Thanks!
The issue with this, is it takes some time to find this is the problem in the first place (ruling out other possible hardware/driver issues). If we’re having to help users properly set their PYTHONPATH, the bug report ends up being technical support, taking time away from development.
What exactly does the last part, about os.environ, mean? It seems the new Blender flag directly sets the Python Py_IgnoreEnvironmentFlag, which is described as
Ignore all
PYTHON*environment variables, e.g.PYTHONPATHandPYTHONHOME, that might be set.
But that is presumably during interpreter initialization. So the remark about os.environ means that PYTHONPATH can still be updated in a script and this will work as usual?
paulm@cmstorm 09:59:~$ touch /tmp/mod.py
paulm@cmstorm 09:59:~$ PYTHONPATH=/tmp python -c 'import mod'
# -E ignores PYTHONPATH
paulm@cmstorm 10:00:~$ PYTHONPATH=/tmp python -E -c 'import mod'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ModuleNotFoundError: No module named 'mod'
# Circumvent -E via sys.path
paulm@cmstorm 10:00:~$ PYTHONPATH=/tmp python -E -c 'import sys; sys.path.insert(0, "/tmp"); import mod'
paulm@cmstorm 10:00:~$
Your Python scripts can still access environment variables and do what it likes with them, it’s just the environment variables documented in python --help will be ignored by the Python runtime.
If you want to read in the PYTHONPATH and add it to the sys.path you could, however if you know what you’re doing and want to have the previous behavior, suggest to use --python-use-system-env.
Every design decision is a compromise. In this case, we chose to make Blender work properly for the naive user while people doing clever things have to be just a little bit more clever. It is not a bad trade-off.
It might be worth noting that using -- on the command line stops argument parsing and everything after that will end up in sys.argv .
So how would we get blender working with custom paths appended to PYTHONPATH? I see a lot of answers saying it was a problem for new users but haven’t found any working examples for experienced users.
For example, I have a tool that uses a venv and launches software from a GUI (Maya, Houdini, Nuke, etc). I’m able to launch every program with different versions of Python just by setting PYTHONHOME and add paths to PYTHONPATH env variables on launch, but this doesn’t seem to be allowed in Blender. I’ve found that adding paths to sys.path in my custom startup script works, but I’d rather pass env vars during subprocess.Popen.
Example: I’m trying to pass Qt dependencies so my script can call for them from Blender. Here’s what I’m trying with Blender:
# variables set up ^^^^^^
# Edit envs
_envs = os.environ.copy()
_envs['BLENDER_USER_SCRIPTS'] = scripts_path # This works and finds my custom startup.py script
_envs['PYTHONPATH'] += 'custom/abs/path'
# Launch software with custom envs
subprocess.Popen([exe_path, file_path , --python-use-system-env], envs=_envs )
What needs to be added to PYTHONPATH and what do I need to set PYTHONHOME to in order to use Blender’s python with a few paths added to it? Is this possible? Thanks!
A recent change was made to ignore Python system environment variables, like PYTHONHOME, by default. Call blender with --python-use-system-env to enable this again. I.e. from blender -h:
--python-use-system-env
Allow Python to use system environment variables such as 'PYTHONPATH'.
Edit: sorry, missed that you already were passing this option. Is your Popen example above not working? Note that the order of arguments to blender matters, so perhaps your script in file_path is trying to access the dependencies, but they only get added to Python paths after --python-use-system-env has been executed. Might that be the issue?
Yes, my above example isn’t working. Its a little pseudocody, but it’s basically what I’m doing–trying to edit PYTHONPATH and passing that into Popen env for Blender.
exe_path (blender exe location) and file_path are just strings so I don’t think there’s an issue there. I think I need to insert Blender’s default PYTHONPATH in the front of the PYTHONPATH. Thats what I ended up doing to get Maya to run correctly because it kept trying to use my version of python instead of its embedded version. I also had to set Mayas PYTHONHOME env var.
Do you know the Blender values for PYTHONPATH and PYTHONHOME? Like what paths does Blender need to run properly? I took a look and found python.dll (I think I need the python.exe though) in the install dir and the site-packages folder in python/lib. I’m not at my computer to test, but I’m thinking those directories would be my starting point.
What exactly do you have set in PYTHONPATH and PYTHONHOME? Blender should not need anything in those variables to use its own builtin interpreter. PYTHONHOME should be left alone as much as possible and preferably empty. And it’s only for modules loaded from Python code within Blender that PYTHONPATH can have an influence, but only when --python-use-system-env is set:
# PYTHONPATH ends up in Blender's sys.path
melis@juggle 23:29:~$ PYTHONPATH='/tmp/doh' blender -b --python-use-system-env --python-expr "import sys; print('/tmp/doh' in sys.path)"
Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-04-24 12:06:14)
Read prefs: /home/melis/.config/blender/2.82/config/userpref.blend
True
# PYTHONPATH does NOT end up in Blender's sys.path
melis@juggle 23:30:~$ PYTHONPATH='/tmp/doh' blender -b --python-expr "import sys; print('/tmp/doh' in sys.path)"
Blender 2.82 (sub 7) (hash 375c7dc4caf4 built 2020-04-24 12:06:14)
Read prefs: /home/melis/.config/blender/2.82/config/userpref.blend
False
But maybe I’m misunderstanding what the issue is you’re seeing. What error/warning do you get exactly?
I’m an idiot, I temporarily put a pass under an else: before the PYTHONPATH logic so it wasn’t getting around to setting that on execution. I also read that the scripts path needs the modules and addons folders along with the startup folder so I added those. It’s working now.
My subprocess:
app_session = subprocess.Popen(
[
app_exe,
file_path_abs,
'--python-use-system-env'
],
env=_envs,
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP
)