VSE and Sound state / future

For the visual modifiers, the modifiers of the active strip can be copied to the selected strips. To keep consistency, this could also be implemented for audio modifiers down the road.

2 Likes

It makes sense for consistency.

Hi, i am looking at the implementation of this EQ within the audio standards.
Audio_compositing_nodes

I dont have anything against a modifier, but people who use to edit audio, EQs in this situation, will look at nodes first, as a compositing nodetree.
Of course, this is not without consequencies, because it needs an input audio clip to be added, and an output (file / mix)… but after all, in the sens of “all nodes”, i suppose this option is a step forward.

Anyway, i can try to look at a basic nodes setup (input > gain > output) before going for more advanced audio treatments, Eq is one.
I dont know exactly what the BF has accepted as an audio upgrade, but why not checking here that both, modifiers and nodes and coexist.
thx

4 Likes

Hello,
I’m very glad to see that there is interest and activity related to sound in Blender.

The good and bad news are: as I have written @iss and @sergey , I have understood that the both agree that Blender could have some extra audio features, but not too many. I would say that they prefer that most audio tasks would be done mainly in an audio application, and use Blender- VSE features as last touchs. Sergey insisted in the UI/UX to be well defined.

So, the challenge or task would to:

  • define a short list of useful features (from the most to less)
  • define their UI/UX

I am a noob in Blender source code, but with my work in the last months I’ve seen that audio is integrated in a very different way to the rest of modules in Blender, which makes it a little difficult to add something. It shares the problems with the “strips” not been ID blocks (@tintwotin has talked about this several times) but with the extra problem of dealing with Audaspace integration.

For example, what I thought as a very simple patch to show the samplerate and channels of a sound strip, it is becoming something really complex because of these odities.

Are we in time to involve this in GSoC 2022?

3 Likes

This will be great to have sound effects like node.
While blender developers won’t to have many audio effects inside blender, we user want have a great experience with the little number of effect they accept.

Here is a list of my wish effects:

  • EQ
  • REVERB
  • DELAY
  • COMPRESSOR

@pistolario If someone can help you implement them as a node please let him do it please.

@Uriel_Deveaud The main problem here isn’t the lack of interesting ideas or submitting patches, the problem is to get patches with new ideas actually committed. In order to get it committed, you’ll have to convince the hired BF devs Sergey and F. Siddi. ISS is doing 99% of the work in the VSE module, but apparently, does not have deciding power on new features. Take notice on how much time and work Pistolario have put into the audio development, without getting a clear answer on the EQ is a wanted feature or not. Basically the only way to get a new feature into the VSE is by finding a feature in another Blender editor and then implement it into the VSE, because consistency is king. But anything outside of this or things needing a new UI design are bound to be dead-end, because there is no UI lead(afaik), and therefore no one to decide on UI changes.

What I’m trying to say is that even though audio nodes would be wonderful to have, it is going to be a long and winding road, which properly will lead to nothing, which is why I suggested that you used your enthusiasm and coding talents to help out with the work of Pistolario, since he has achieved to get some positive dev attention for his audio project and let things build from there.

@pistolario I’m not sure what the conclusions of your latest post here are? Is it back to square one? Is Sergey that unhappy with the UI of your EQ patch, that he is requesting you to start all over with the proposal yet again(if it is, it’s my fault, sorry)? Or is it a general request for additional audio features, to start with proposals that include UI?

In previous years neXyon had a suggestion for an audio GSoC project, but no one stepped up, and I can’t find the suggestion anymore. The trouble with a GSoC project for audio is that neXyon & ISS would have to mentor it(and get S. & F.D. to okay it) , so you would have to try to convince them in investing the time in it. The deadline for GSoC is the 19th of April: Blender has been accepted for GSoC 2022 - #9 by ThomasDinges GSoC - Blender Developer Wiki

1 Like

@tintwotin My sensation is not to be in “square one” again, but that “they” want a fully completed, perfect and polished patch. But in order to get that, I’m not sure if they are really aware of the interest in the community, or if “they” are ready to help or guide in order to get that “full gift”, or worse, “they” could be really afraid of any new feature, because they would have maintain it, and in their inside “they” don’t want to get involved in that.

I talked to @Uriel_Deveaud , and we decided that it would be better to work in 2 features in parallel. He will try to do a VUmeter (a very specific feature, which needs a lot of attention), and I’ll deal with the Graphical EQ.

By the way, yesterday I updated my patch to show simple specs of a “sound strip”: ⚙ D14565 Display sound specs in VSE

Trying to be more useful, I find really difficult to understand the integration between Audaspace and the “depsgraph of Blender”. I tried to ask @iss and @neXyon but I think that they didn’t understand me. Depsgraph makes a “copy on write” of the full Scene in any “strip change”. The problem if that, apparently, the relation between the “Sequence struct” and the “Audaspace sequence struct” is stored in a copy, and not in the “original structure”. Perhaps I don’t understand well how to handle this. In the “sound specs patch”, I add a “search in the depsgraph” in order to get the data but, is that the right way?

Anyway, I’ll try to change the patch with some suggestions of @neXyon , but others suggestions are against “simplicity”. It is going to be tough…

1 Like

hi, looking at the implementation of vu-meters in the VSE. A couple of options are there and i wanted to call your attention on them:

  • per VSE strip (type:audio)
  • per VSE channel (channels 1…128)
  • Master VSE audio Outputs (channel 0)

For the “per channel” option:
Audio_vse_channels
Note: The Master channel (ch.0) has always a space at the bottom of the sequencer window, so master channel settings can show up in this space.

A question remains, “per-strip”… why ? when? how…
I mean, as the volume level is dynamic ONLY when the strip is played in the sequencer, the channel volume is controlling over the single strip volume. And i am wondering if during editing a video (in other softwares), we all apply gain reduction to strips and is this more elegant than Channels mixing?

First of all, It’s great to see some ideas / designs flowing.

If I remember some DAW, I think that there are controls per channel (mute, solo,… even vumeter)
But, is it very Blender style? Or, in the NLA (which would be something similar) is there any concept like that? I think that that would be a little discussion inside the Blender UI group :wink:

I like a lot your right mockup (controls per channel and vertical vumeter), but we would have problems with the channel 0. Now there is no channel 0 in the VSE, so it would be difficult to get it.

I would like to call your attention to another option, more similar to the “Video Sequencer - Preview section”. In that space you can see channel 0 (final composition), or any channel. In the audio world, it would be to see the VUMeter or any other visualization about channel 0 (final), or selected channel. Would it be too complex? That would give more space to see "per audio output (mono, stereo, …).

But, remember that, No matter what you design or implement, that Blender UI group will change it. I would focus in the most simple way to have something that works, and then let them to think about how do they want it.

Nice to see some UI mock-ups. I hope you are in contact with @neXyon? I remember him mentioning that getting a correct master VU meter may take several months to implement. And the meter in VSEQF isn’t correct: https://github.com/snuq/VSEQF

So, in other words: the individual channel vu-meters look great, but there may be serious challenges to just getting a master VU meter working, which properly needs some looking into first.

If the UI implementation for a master volume should be a vertical one like in VSEQF, it could be implemented as a narrow sidebar next to the Sequencer sidebar in the right, or next to the Preview sidebar. A more clumsy way would be to add it as a View Mode in the Preview, but this would demand the user to open up yet another instance of the Preview, just for viewing the VU meters. Clumsy.

The Channel Headers could need a proper design discussion, but no one from the UI team stepped up(!). IMO, are numerous vital functions missing from them.

In relation to your suggestion, a question is if it should be possible to add and manipulate ex. volume, modifier settings, and keyframe settings for the entire channel from the channel headers? Or should that be kept pr. strip? If it is pr. strip maybe the vu meter could be drawn in the background of the volume widget(not a very elegant solution, but not very intrusive UI wise) - or a horizontal float widget on it own?

1 Like
  • Forget about VU Meter per Chanel and per Strip.
    Remember that VSE is not design for only audio Strip. What if a Chanel doesn’t contain a Strip audio? And for sure while editing many Chanel will not contain audio.

  • VU Meter per Strip is acceptable if it will be implemented as @tintwotin suggest it. Having it in panel within Volume

  • MASTER VU Meter could simply be implemented as @tintwotin suggested again near right panel .

Hi,
the UI can be difficult, ugly and will not be ok at the first try.
I think that the first goal should be to capture the global sound from Audaspace and try to paint anywhere the data.

When we have something, about VUmeter and about EQ, I think that Pablo Vazquez could work with it, because he does things sound related and is in the UI Group of Blender.

peace,
i have an idea that is to make EQ and COMPRESSOR etc… as strips effect, and anothers similar strips has a same effects but to all audio bellow (like adjustment layer) and that will give us the ability to effect a part of the audio and not all the audio bellow …
so the visibility of the previous changes that we did by EQ end others will be always visible in channels + a general EQ(=adjustment layer) effect all bellow audios by one go

@hamza-el-barmaki If you code, maybe adding sound to the Adjustment Layer could be an idea? Then Pistolario’s EQ modifier could be added to that(if that eventually will be committed).

As I see, there are several points where to put “sound modifiers”:

  • Modifier of the strip, as a video strip (color, curves, …)
  • effects stript, a special strip used to modify one or more strips as a whole
  • at the end. For example, a “sound compressor” at the end of all the mixing

Any case, now there is no infrastructure to put sound modifiers in any place, out of the strip. It would be a first target. @Uriel_Deveaud had designed something for the mixer, which would be a great place to inject “final modifiers”.

2 Likes

Hi

just a couple of captures to show the progress of the “Sound equalizer modifier”. it uses the data infrastructure of “video modifiers”.
A selector of modifiers (you can add the same several times)

Just the modifier

4 Likes

Awesome! Glad to see this is coming along well.

So cool! Thanks a lot for the hard work!

I’ve updated the patch in the web.
Now it’s time to revisit pending issues:

  • UI / UX. Now it creates always 2 graphs. Internally it could have any number of graphs, but when or how to configure the number of graphs and the ranges? Others:
    • Remove the “Sound” from “SoundEqualizer”
    • Add the “Hz” in the ranges
  • other developers comments: Nexyon and ISS will have something to comment
  • performance. Some operations are not optimized and are very likely to improve
  • dependencies. I’ve written to LazyDodo (I think it’s written so), and he’ll contact with people in charge in order to get FFTW in single precision in Blender (now is double, used by Ocean sym)
  • sound quality. Now it’s a linear filter. I think that this could improve with Minimum phase filter.
  • errors
2 Likes

The patch is evolving. Suggestions by Nexyon have been incorporated, and now I’m dealing with suggestions by ISS.
One thing that we should concrete is the way of grapical equalizations we permit.
Now, suggested by Tintwotin, you can select one of 3 predefined pre-sets of graphics (1, 2 or 3 graphics).
Furthermore, by API you can destroy all the graphics, and you can ADD one graphic

My problem is with the UI of the graphics. I don’t know how:

  • force limits in the graphics
  • when the user selects reset graphic, it goes always to a ramp from minimum to maximun

Suggestions?

2 Likes