Hi everyone, this is a feedback thread for my GSoC 2022 project. To keep weekly report thread clean, I don’t want to write too much details in that. Instead, I will write more in this topic(maybe just like write blog here? I’m not sure). I hope to get more feedback to improve my work and code ability. Feel free to use this thread!
Weekly Report(more details):
I am busy with my final now, for I can’t do much work these days.
What I have done:
Read the source code of VSE
I have read some source code of VSE and I found some files maybe important to me. BKE_sound_read_waveform function in sound.c reads the audio and saves it in sound->waveform. The core of this function is AUD_readSound function in AUD_special.cpp, which actually read the aduio and waste most time of waveform drawing in my test. AUD_readSound create a reader to get minimum signal, max signal and power of the signal( I tend to use signal to explain audio volume ). These three value will be saved in buffer which is a float pointer parameter of AUD_readSound.
The returned reader also interest me.
std::shared_ptr<IReader> reader = ChannelMapper(*sound, specs).createReader();
ChannelMapper seems related to a factory method but I’m not sure.
Waveform drawing function is draw_seq_waveform_overlay in sequencer_draw.c. It use waveform data to draw waveform. But I don’t know the algorithm that it use.
questions:
None
What to do next week:
-
Learn more about factory method and check whether
ChannelMapperis related to a factory method -
Read the source code of
draw_seq_waveform_overlayto understand the drawing algorithm
4 Likes
Weekly Report(June 20 - June 26):
I am so busy with my study this week that I have no time to implement my plan. I will certainly restart my job next week.
Weekly Report:
I read the source code of audaspace, BLI_thread and waveform drawing this week and implemented a demo in my local test branch. Though my demo has many bugs, it improves the speed of waveform drawing by 22%!
The mainly work I have done in the demo is parallelizing the loading. VSE used to load waveform one by one, that is, it will not load other sound until the current one have been loaded. In my demo, every sound has individual load thread, it will make full use of disk I/O and imporve the speed of waveform drawing.
This video may not be very clear as the original video is quite large so I can’t upload it.
4 Likes
Weekly Report(7/04 - 7/10):
My code is commited to soc-2022-waveform-drawing branch which is mainly on sequencer_preview.c. It becomes more robust compared to last week’s and I have test it in the case when loading many large audio.
The benchmark I made last week is not quite reliable for there are too many background applications running in my computer. In my new benchmark, the speed of drawing improves by around 37.6%. I think this result may be more reliable than last week’s.
Next Week:
The next week I will try to implement waveform drawing system which tracks timeline playhead. The final effect is just like what Figure(a) shows. In this system, Waveform will be processed in segments. Those near the timeline playhead should be guaranteed to be drawn preferentially, which makes sure that artists view waveform “in real-time” as drawing is being processed.
This will lead changes in BKE_sound_read_waveform function.
6 Likes
Weekly Report (7/11 - 7/17):
I try to develop the feature mentioned last week, but haven’t finished yet. I have met some technical problems.
To develop it, BKE_sound_read_waveform should be changed. AUD_readSound is the core function of BKE_sound_read_waveform which reads the actual waveform data. In my plan, I will use two threads to call this function and each of it will read a part of sound and write waveform data to sound->waveform. This requires AUD_readSound seekable so that we can read diffrent part of sound. But sadly it isn’t. So, I add a new function AUD_readSoundSeekable in AUD_Special.c. It has a parameter that can determine where to start to read. It mainly rely on Reader’s seek(seek_pos) funtion. Seek function should take effect but it seems that it doesn’t seek the audio. I have no idea about it now.
Besides, for the seek function will reduce the audio reading speed, I am not sure wheather it will boost the load speed.
I will discuss these two problems with my mentor next week. And the feature development will be certainly finished next week unless it is not “cost-effective”.
2 Likes
Weekly Report (7/18 - 7/25):
I am sorry to not finsh my goal made last week. An other project of mine takes me too much time and I get ill this week. I’m learn ffmpeg now, hoping to solve the seek problem.
1 Like
Weekly Report (7/25 - 7/31):
A planned trip made me not have much progress this week, but I tried to develop segments drawing. I have wrote the segements drawing demo and it is testing now. Still have lots of bugs.
1 Like
Are you still working on this project?