GSoC 2021 Draft Proposal: Faster Import & Export for STL & PLY Formats

Contributing to Blender Summer of Code
1

Hi everyone, as a followup of this thread here is my proposal draft:
GSoC 2021 Proposal - Faster Import & Export for STL & PLY Formats

Pinging @Howard_Trickey @ideasman42 @jacqueslucke since they’re listed as possible mentors for this project.

Feedback is -as always- welcome.

Name

Ahmet Sait Koçak

Contact

Email: [email protected]

Blender Devtalk: ahmetsait
Blender Community: ahmetsait
Blender Chat: ahmetsait
Bug Tracker: ahmetsait

GitHub: https://github.com/ahmetsait
Blog: https://ahmetsait.github.io/blog

Time Zone: GMT+3

Synopsis

A crucial part of Blender’s functionality is its ability to work with a wide range of file formats. This is essential for interoperability with other tools since different software has different levels of support for various file formats. Blender does support many of the common (and some not-so-common) formats out of the box, except there is a catch: The current import & export code for STL and PLY formats are rather inefficient in terms of both speed and memory. A lot of work has been done on the OBJ format in the past year’s GSoC, and this project aims to build up on that work to leverage implementing efficient import-export functionality of STL and PLY formats.

Benefits

Huge time savings for anyone dealing with millions of vertices or objects. Many people have limited control over which format they can use and making Blender tolerable for IO intensive tasks is an enabler to various areas such as 3D printing.

Deliverables

  • Native import-export plugins for STL and PLY
  • Automated benchmarking and testing code
  • Various benchmark results as well as documentation

Stretch Goals:

Project Details

Profiling of the add-ons for Suzanne mesh with level 5 subdivision surface modifier (~500 000 vertices) yields us the following results:

PLY Binary Export 18.813s
PLY Binary Import 39.250s
PLY Ascii  Export 21.063s
PLY Ascii  Import 41.562s
STL Binary Export  5.109s
STL Binary Import  6.312s
STL Ascii  Export 15.016s
STL Ascii  Import 17.109s

Some highlights from the profiler:

  • List comprehensions result in excessive copying and are a significant slowdown
  • PLY exporter rounds vertex normals and UVs for no apparent reason
  • Struct packing / unpacking is a surprising time consumer which would be no-op in C/C++

Python being an interpreted language naturally contributes to the overall speed issue. This project aims to rewrite the PLY and STL IO code in C++ with a design that follows other current IO modules USD, Alembic, Collada and OBJ.

Key points to follow in implementation, some also mentioned in the past years’ GSoC proposal:

  • Accessing memory in a cache friendly way
  • Reducing memory allocations and copying, using slices where plausible
  • Proper buffering, minimizing the memory <-> disk sync

There are also some optimizations such as multi-threading and vectorization that are likely not necessary but would be interesting to experiment if the time constraints are not a problem.

Project Schedule

Important Notes: My exams will be over on June 26 according to my university academic calendar, and coding period of GSoC this year will start on June 7, meaning that I should be getting my hands dirty in the community bonding period. There is also a high chance of me doing internship in the summer but GSoC this year is less work intensive (and so is the proposed project) to allow such flexibility.

Week 1-2 (June 7 – June 20): Getting familiar with Blender code base; DNA, RNA, BMesh, asset pipeline and the general development workflow. Studying other IO modules and past years’ GSoC branch to carve out a design that my mentor approves.

Week 3-4 (June 21 – July 4): STL import & export

Week 5-6-7 (July 5 – July 25): PLY import & export

Week 8 (July 26 – August 1): Testing, bug fixes and documentation

Week 9-10 (August 2 – August 15): Wrapping up the IO work and getting it into mergeable state. Tackling drag & drop support for importers if things are done early.

Last weeks act as a buffer for the potential inactivity of the first two weeks and for any possible latency on the current work schedule. I can work on my stretch goals as a backup plan.

Bio

I’m Ahmet Sait, a 4th year Computer Engineering student from Turkey. I started learning programming in my first year in high school and I’ve been doing it ever since, ranging from various hobby projects of mine to contributions to other foss projects. Open source lines up perfectly with my belief in sharing knowledge for the benefit of everyone, and this is reflected in nearly all my projects on GitHub.

Aside from being a Blender user, I have good grasp on 3D concepts and how a lot of things come together, as well as hands on experience using modern OpenGL. I enjoy reading blog posts about graphics programming and hanging out on OpenTK discord server.

Blender is among the most impactful OS projects I had the pleasure to deal with, and a journey to becoming part of this awesome community as a contributor would be amazing.

Some Projects I’ve Contributed to:

  • OpenTK: A Fast, low-level C# wrapper for OpenGL, OpenAL & OpenCL
  • DRuntime: Low level runtime library for the D programming language
  • GFM: 3D maths for D game-dev
  • ASDF: Worlds fastest json parser geared towards processing huge amounts of JSON

Some Projects I’ve Created:

  • BindBC-HarfBuzz: HarfBuzz Bindings for Dlang
  • Binding-Dynamizer: A command line tool for converting D language static bindings into BindBC compatible dynamic ones
  • LF2-IDE & IDL: A fully featured visual game modding toolkit for LF2
  • Dir-Sync: A convenient graphical tool for recursive file system tree differentiation & synchronization
14 Likes