Final Report
Demonstration
The video below used Sample Sound node and simulation zone together, forming a 3-D waterfall plot.
The project file is attached in the description of my working pull request. The audio is made by myself and licensed under CC0.
Recap of the Goal
This project aims to:
Provide the ability to retrieve sounds from files in Geomety Nodes,- Generate amplitude/frequency response information based on several customizable parameters, and
- Be written in native C++ with caching/proxy operations to speed up execution.
State of Deliverables
Listed in Proposal
-
A new socket type in Geometry Nodes called Sound that corresponds to Blender’s data-block type of Sound;
-
A Sound Input node that can read a single Sound from Sounds in data-blocks;
-
A Sample Sound node that takes a Sound, which then goes through several tunable internal processes, including:
Gain control,(this could be easily handled by user,)- Playback progress (sample time) control,
- Temporal smoothing,
- Audio channel selection,
- Frequency specification (FFT.)
The Sound will be finally converted to a corresponding amplitude/power value (as Float) by using the options above.
-
WIP A series of usage examples and documentation related to all the deliverables listed above.
Extra ones
-
A Sound Info node that allows user to get audio channels and sample rate from a sound.
-
New unit of frequency Hertz (Hz) and Kilohertz (kHz) for expressing pitch of a sound.
Pull Requests
| Deliverable | Link | Status |
|---|---|---|
| 1, 2, 3 and 5 | blender#122228 | Open |
| 4 | blender-manual#104897 | WIP |
| 6 | blender#125915 | Merged |
Use Cases
The most common use case would be visualizing spectrum from a sound:
If combined with simulation zones (as demonstrated by the video at the beginning), it’s able to create a waterfall plot from a sound:
Next Steps
- Refine the manual and provide more usage examples for nodes listed above.
- Consider whether the Sample Sound node needs to have a “Bake” option that saves all amplitude values into file. For now, it seems that the computation is efficient enough.
- Keep fixing bugs and improving functionality until release!
What I Have Learned
Beside from basic coding skills, I’ve gained another two invaluable experiences.
Firstly, think before you code. This is probably the most important thing I learned in this summer. Coming up with a decent design is almost always harder than writing actual lines.
Also, don’t be afraid to ask. Although it’s a pity that I’m probably not the most suitable one to speak out this sentence (actually it’s my mentor who initiate a conversation most of the time…) I’ve already realized the importance of exchanging opinions and problems—it helps us to find the best way to implement everything, and that’s what makes a great software.
Last but not Least
I want to express my sincere gratefulness to my mentor, Mr. Jacques Lucke @jacqueslucke, for his constant support and supervision on this project. The last and hardest part to make the node able to be used in production, i.e. the cache, is also based on his solution. I also appreciate everyone who helped me during this summer. Without them, this project won’t be as complete as it is by now.
While it’s the end of this year’s GSoC, it’s not the end of my journey in Blender. I’ll continue to do my part on this project and the rest of Blender’s components, for it’s continuous contribution to the world of open source (creative) software.