Dear Fellow Scholars, this is Two Minute Papers with Károly Zsolnai-Fehér.
This work is about building an AI vocoder that is able to synthesize believable singing
from MIDI and lyrics as inputs.
But first, what is a vocoder?
It works kinda like this.
Fellow Scholars who are fans of Jean-Michel Jarre's music are likely very familiar with
this effect, I've put a link to an example song in the video description.
Make sure to leave a comment with your favorite songs with vocoders so I and other Fellow
Scholars can also nerd out on them.
And now about the MIDI and lyrics terms.
The lyrics part is a simple text file containing the words that this synthesized voice should
sing, and the MIDI is data that describes the pitch, length and the velocity of each
sound.
With a little simplification, we could say that the score is given as an input, and the
algorithm has to output the singing footage.
We will talk about the algorithm in a moment, but for now, let's listen to it.
Wow.
So this is a vocoder.
This means it separates the pitch and timbre components of the voice, therefore the waveforms
are not generated directly, which is a key difference from Google DeepMind's WaveNet.
This leads to two big advantages: One, the generation times are quite favorable.
And by favorable, I guess you're hoping for real time.
Well, hold on to your papers, because it is not real time, it is 10-15 times real-time!
And two, this way, the algorithm will only need a modest amount of training data to function
well.
Here, you can see the input phonemes that make up the syllables of the lyrics, each
typically corresponding to one note.
This is then connected to a modified WaveNet architecture that uses 2-by-1 dilated convolutions.
This means that the dilation factor is doubled in each layer, thereby introducing an exponential
growth in the receptive field of the model.
This helps us keep the parameter count down, which enables training on small datasets.
As validation, the mean opinion scores have been recorded, in a previous episode, we discussed
that this is a number that describes how a sound sample would pass as genuine human speech
or singing.
The test showed that this new method is well ahead of the competition, approximately midway
between the previous works and the reference singing footage.
There are plenty of other tests in the paper, this is just one of many, so make sure to
have a look.
This is one important stepping stone towards synthesizing singing that is highly usable
in digital media and where generation is faster than real time.
Creating a MIDI input is a piece of cake with a midi master keyboard, or we can even draw
the notes by hand in many digital audio workstation programs.
After that, writing the lyrics is as simple as it gets and doesn't need any additional
software.
Tools like this are going to make this process accessible to everyone.
Loving it.
If you would like to help us create more elaborate videos, please consider supporting us on Patreon.
We also support one-time payments through cryptos like Bitcoin, Ethereum and Litecoin.
Everything is available in the video description.
Thanks for watching and for your generous support, and I'll see you next time!
For more infomation >> INSANE INFINITE BLOCK DUPLICATOR (minecraft 1.12.2, use subscription) - Duration: 2:31. 
For more infomation >> 王宝强离婚身陷舆论怪圈，儿子竟说出王宝强和范冰冰的关系！ - Duration: 7:25. 
For more infomation >> Última hora sobre el estado de salud de María Teresa Campos - Duration: 3:09. 
For more infomation >> Jacqueline Benoit : « Même le chien Laeticia Hallyday s'en est débarrassée » - Duration: 3:19. 
For more infomation >> Gad Elmaleh : Ac**sé de plagiat, il s'explique dans Quotidien -[Nouvelles 24h] - Duration: 2:51. 
For more infomation >> Volvo V50 1.6D S/S SPORT * NAVI PARKEERHULP BLUETOOTH * - Duration: 0:59. 
For more infomation >> 大S汪小菲一家春節出遊，汪小菲抱著熟睡女兒讓老婆悠閒逛街 - Duration: 4:12. 
For more infomation >> The new Mercedes-Benz C-Class 2018: World Premiere | Trailer - Duration: 0:47. 
For more infomation >> Digital Learning Day: Highlighting Tennesse's Online Public School based in Bristol - Duration: 1:31. 
For more infomation >> The new Mercedes-Benz C-Class 2018: World Premiere | Trailer - Duration: 0:47. 
For more infomation >> A Los Angeles, Laeticia Hallyday vivait cloîtrée - Duration: 3:13. 
For more infomation >> Hyggestream - Duration: 53:20. 
For more infomation >> Holly Stout (Global Rally 2017.) Prezentacija nove linije za njegu kože - Duration: 8:34. 
For more infomation >> A Los Angeles, Laeticia Hallyday vivait cloîtrée - Duration: 2:56. 
For more infomation >> Pourquoi Johnny Hallyday a perdu 10 millions d'euros - Duration: 2:27. 
For more infomation >> Fiat Ducato 30 2.3 MultiJet 130pk L1H1 33% Korting! - Duration: 0:54. 
For more infomation >> A Los Angeles, Laeticia Hallyday vivait cloîtrée - Duration: 2:39. 
For more infomation >> Demi Lovato : Sa collaboration avec Luis Fonsi lui offre un record de carrière - Duration: 2:06. 
For more infomation >> COMMENT S'ÉCHAUFFER EN MUSCULATION | EN 60 SECONDES - Duration: 1:22. 
For more infomation >> Mighty No. 9_テクニカルボーナス狙い 再挑戦 電波塔 S RANK RAY（レイ） - Duration: 7:27. 
For more infomation >> Sebastián Yatra - SUTRA
For more infomation >> 7 PAYS SUSCEPTIBLES DE DEVENIR DES SUPERPUISSANCES D'ICI 2050 - Duration: 10:22. 
For more infomation >> Johnny Hallyday « détestait être attaqué sur les histoires d'argent » - Duration: 3:34. 
For more infomation >> Pourquoi Johnny Hallyday a perdu 10 millions d'euros - Duration: 2:20. 
For more infomation >> BMW 3 Serie Touring 316D 2.0D EXECUTIVE AUT8 | Navi | Sportstoelen | Xenon - Duration: 0:54. 
For more infomation >> Volvo V50 2.0D Edition II - Duration: 0:59. 
For more infomation >> Mitsubishi Outlander 2.0 DI-D Invite 4x4 - Duration: 1:01. 
For more infomation >> Johnny Hallyday « détestait être attaqué sur les histoires d'argent » - Duration: 2:42. 
For more infomation >> La vérité dégoûtante au sujet des bars à ongles - France 365 - Duration: 6:58. 
For more infomation >> Quel drôle d'animal dans le lac Beaulieu - Duration: 4:05. 
For more infomation >> Affaire Johnny Hallyday : Le gros coup de gueule d'Henry-Jean Servat - Duration: 2:22. 
For more infomation >> Passei mal na Itália - Frases úteis em italiano - Duration: 2:07. 
For more infomation >> Special na 10 subów!!! Śpiewam przez twe oczy zielone - Duration: 6:11. 
For more infomation >> Vitamin D3 mein Selbstversuch 10000 i E + Blutwert - Duration: 13:05. 
For more infomation >> TO YOU FOR ALWAYS MY LOVE [SHORT POEM] - HD - Duration: 1:04. 
For more infomation >> Unreal Engine 4 - Vertex Paint - Duration: 6:57. 
For more infomation >> PRAI Ageless Throat Decolletage Creme 6.8 fl. oz. in F... - Duration: 18:31. 
For more infomation >> PRAI Ageless Throat Decolletage Creme 6.8 fl. oz. in F... - Duration: 11:58. 
No comments:
Post a Comment