Lib2to3 missing , on purpose?

The assumption you make is that these tools will work or give anything like a useful migration to Python3.

In practice I would guess this “half-works” but the time you spend messing with remaining broken things you could better spend on upgrading your project to Python 3 properly.

Further, why would you rely on this unofficially supported “past” module?
It had a single release 4 years ago.

Probably it’s already not well supported - so you only postpone the inevitable.

Guess, that’s the whole point, postpone the inevitable, but the time gained, is what is needed to have tools working in both contexts.

In practice I would guess this “half-works” but the time you spend messing with remaining broken things you could better spend on upgrading your project to Python 3 properly.

Tools has to work in both, or there’s no point now of doing a full port to python 3, as you are the only dcc application supporting it.

Let me try to ask something different then. how can I build blender with this module in so at least I can test ?
And please, can anyone let me know what problem or harm could cause to blender ?

Thanks.
L.

p.s
if past is not the answer , do you have any other module you can suggest, which can allow me to load python3 code in 2 and viceversa , I’m more than happy to change if there’s anything better !

Try copy-paste install python3 and copy-paste the module into Blender’s Python directory.

No really the solution I was after when I first posted in this forum, but if this is as far as I can get, fair enough.
This way I won’t be able to officially support blender, which is a shame.

I don’t want to be pedantic, but I’m still in need of knowing what harm can cause to blender having this module back in, as I might face this somewhere else.

Cheers.
L.

This is not a final solution, only so you can test the modules to see how well they work, then get back to us with your findings.

Cheers, I’ll do so then, and get back to you.
Thanks for you time.
L.

Hi @ideasman42,

haven’t finished off testing fully (it’ll take some time) but so far, here are some results:

  • I’ve been installing lib2to3 in my system python3.6m installation (pip3.6m install lib2to3)
  • took the module and copied to ./2.80/python/lib/python3.6
  • installed pip in ./2.80/python/bin/python3.6m with get-pip.py
  • use new ./2.80/python/bin/pip3.6 to install future module (which bring past)
  • use blender python to install (python3.6m setup.py install) my module.
  • open blender → import module → intialise classes → success :slight_smile:

So far , apart for some minor issues i need to nail, seems all to be reflecting the testing results we have in place.

I’ll be able to give you more information in a while (free time permitting), but so far all seems to be working fine.

Cheers.
L

Hi @ideasman42, finally managed to get some time to look into this today, and I managed to get all working (yay!). Actually, as hoped , the ability to import and run the code from within Blender , allowed me to spot issues I did not found with the unittests.

I’m not sure what’s the next step at this point, but having this module back , is a real game changer to me.

Cheers.
L.

p.s
checked now on the sources and found this : https://git.blender.org/gitweb/gitweb.cgi/blender.git/blob/HEAD:/source/creator/CMakeLists.txt#l567
https://git.blender.org/gitweb/gitweb.cgi/blender.git/blob/HEAD:/source/creator/CMakeLists.txt#l864

removing these lines is enough to re enable it ?

@ideasman42, have you had any further thought about this ?
asking as I have seen you are currently working on reviewing python in 2.8.

Thanks.
L.

Hi Blender devs, hi Lorenzo!

I’m also giving my vote to including lib2to3 until vfx platform officially switches to python 3x (pushed to 2020 unfortunately…). These kind of small things are vital for pipeline tools like Ftrack or Shotgun to work with Blender. If Blender doesn’t talk with (even bad) standards, it just forces companies to leave Blender out of their pipeline. And in the worst case not allow artists to use Blender. Please listen to people when they are trying to help us blender users to get Blender recognized in bigger companies.

Cheers,
M.

1 Like

Am yet to see any evidence this is useful in non-trivial real world scenarios.

Also, blender 2.8x will likely release in 2019, Python2.x will be discontinued in 2020 so the motivation to support something that will be abandon-ware soon after is not very high.

Not to exclude the fact that the python community has moved to Python 3 the last few years for good.

And this is clearly illustrated in this link

https://python3wos.appspot.com/

Red is Python 2 library, green are libraries that ported to 3 , when Blender first embraced Python 3. Back then the site was named “Wall of Shame” and it was almost all red.

By the way its possible to use Python 2 with Blender and is not even hard to do. I have done this for using Python with another language, its basically involves running Python 2 script creating a new process and then communicating with Blender’s Python 3 via a IPC method like a socket or if you want something blazing fast a memory mapped file (shared memory).

As ideasman said I would stay way from Python 2 because is basically abandonware and as he said its to be discontinued 2020. Actually even its creator Guido was surprised how long it took the community to fully embraced python 3 , mainly because devs did not want to go the extra effort of rewriting code, which means that Python 2 was on life support for years now.

Nonetheless you can do it yourself and is not that hard.

Hi Kilon thanks for you input.

I just want to make clear that you do not have to convince me,
I’m quite sure on what basis my statements are made, as these are simple , real production needs.

Please do not get me wrong, but your input seems based in a uneducated guess on what the professional vfx industry processes and its needs are. Other industries (eg : game, design etc…) might be a different subject, and properly fit your view, but as I’ve been stated from the beginning what my industry aim is.

We can always found workaround, when proper solutions cannot be found.
This thread is about just this, finding the right solution, or at least trying to.

This said, I’ve got a separate chat with Campbell, and I’ll provide an example as soon as possible of what I’m really in need.

Cheers.
L.

Hi Campbell,

few statements to kick off so there’s no further confusion.
I’ll also skip any big picture rant as I think by now is more than clear what’s my goal is (so I kindly ask you to do the same)

  • I’m not after any automatic conversion of the code, as it would not be reliable for production purposes.
  • I do not need 2to3 executable to auto convert the code for the above reason.
  • I’m up for any option in order to have python2 code working in python3 and viceversa, this is just happen to be the one officially supported by python.
  • lib2to3 is a library which is part of the python base code, not a third party library or module.
  • future module can be installed through pip without lib2to3 but the past module in future won’t work without it.
  • I’m aware of all the disclaimers and caveats about lib2to3

That said,
the main purpose of the lib2to3 for me is the ability of bringing in compatible python object which can work in both py2 and py3, this is supported by the future module in conjunction with lib2to3 through the past module. So my main goal here is to fully be able to install and use:

  • future
  • past

As one of the big changes in python2 vs python3 is how strings are managed one of the goal for me would be to be able to import and use :

from past.builtins import basestring

the same goes for the opposite:

from future.utils import native_string

which leverage some of the differences.

as well as :

from future import standard_library
standard_library.install_aliases()

which leverage some import modules.
(you can read something more about it here : future.standard_library — Python-Future documentation)

These just to show some more obvious.

Long story short.

i’m just after a way to import and use py2 objects in py3 (and viceversa), so the code can happy works in both :slight_smile: and this cannot be happening without lib2to3 back in python.

Cheers.

L.

ps.
I’m happy to contribute with a PR if needed.
Cheers

So you want to access a run-time type that could help you load Python2 code without transforming it…

From reading lib2to3 docs, it focuses on transforming existing code, this seems to be it’s only purpose.

If you don’t want to access parts of the past module that transform Python code at runtime, you can stub it out.

def lib2to3_stub():
    import sys
    modules = sys.modules
    from types import ModuleType
    def stub(modname, **ns):
        modules[modname] = m = ModuleType(modname)
        m.__dict__.update(ns)
    stub('lib2to3', refactor=Exception)
    stub('lib2to3.refactor', RefactoringTool=Exception)
    stub('lib2to3.pgen2.parse', ParseError=Exception)

Example use:

lib2to3_stub()

import past

from past.builtins import xrange
for i in xrange(10):
    print(i)

from past.builtins import str as oldstr
print(oldstr(b"Hello World"))

@lorenzoangeli does stubbing lib2to3 work for your project?

Hi Campbell, thanks a lot for looking into this.

Although I fear won’t fly for an official inclusion (why should I be stubbing something out of a module which is supposed to be bundled with the interpreter?), might be a way for me to keep looking into into blender.

So, not really a viable solution for what I’m after, but might be enough for me to play with it, although will never result in anything official.

As you can guess by now, the only solution I’m really after is to have lib2to3 where it belongs, in the python installation.

Cheers, and thanks anyhow for finding the time.
Happy Blending.
L.

p.s

Some day, someone, will be able to tell me what problems might produce having this library back and what issues has been wanted to be solved, removing it ?

I don’t see how this relates to official inclusion:

If you want to us past from Blender, you can:

  • Load the lib2to3 stub.
  • Load some_module_that_uses_past.

The lib2to3 stub can be done only when used inside Blender.

Also, I’m not sure why you would consider using a stub for an unused module so unreasonable. One could argue that developing new code that relies on an alpha-release from 2014 with 137 open bugs - is the risky part of your stack.

Hi ,
this is fine by me for keep playing with the code.
Thanks !
L.

Chirping back in to say by the end I got rid of the “past” module as suggested and replaced with six, as was also breaking in various other apps .

Cheers.
L.