Topic: Eliminating crowd noise, with artificial intelligence?

[Pittylabelle]

Hi there,

last weekend I taped the concert of Kraftwerk in Bonn, Germany (08 28 2022). The concert was unseated.

I'm sorry to say, I've never experienced such an unfocused, nervous and babbling audience as at this concert. I don't know if it was due to the long Covid break, this concert was also postponed twice, but maybe people are happy just to get back among other people. But it was no fun to listen to it let alone record it.

About three quarters of an hour before the show I looked for a good standing place in the area in front of the stage. Shortly before the concert I then noticed at some point behind me a group of (sorry) "housewives" who had obviously strayed to this concert of Kraftwerk. Then the crowd concentrated in the area in front of the stage, where I was standing. At some point, however, it was so full that I no longer had the opportunity to place myself somewhere else. This group of women behind me actually talked loudly for the whole two hours, as if they were sitting in a cafeteria with background music. Especially in the quiet passages this was unbearable - in the loud passages was then of course shouted.

All the babbling can of course be heard on the recording, and in such a way that it is really disturbing.

This recording has now led me to deal with the attenuation or elimination of audience noise. I haven't even tried that before because I thought it would be quite difficult and time consuming or almost impossible.

I own the program iZotope RX 9 and experimented a bit with it. My idea behind it was: If there is the possibility to isolate only the vocals (or human sounds), then it should be possible to get a track without vocals in the opposite sense. However, with the tools "Dialogue Isolate" and "Music Rebalance" this worked only moderately, even if I only select the relevant areas in the WAV file. The tools does indeed find human sounds, but only very litte.

Then I came across the website of www.lalal.ai. These relatively new possibilities to split tracks into the different instruments seem to be quite popular with DJs at the moment.

I then uploaded a few samples there with a blatant example and I have to say that I was quite flabbergasted at how good the algorithm used there is. The babble was filtered very well, except for a few pieces of music.

Sure, you can't use everything that this algorithm then renders from the files, since of course it also includes the vocals, and there will still be a lot of editing work to be done, but unfortunately the prices for a completely rendered concert are not what I like either imagine.

Has anyone here had similar or good experiences with eliminating audience noise, especially with these new methods of artificial intelligence?

Cheers

Michael

[nulldogmas]

I have absolutely used Music Rebalance in iZotope RX to cut out chatty crowd members — the main drawback is that you can't use it in sections of the music with vocals, or it will delete those as well.

Have you tried futzing with the Separation slider? Usually there's a setting somewhere there that will at least reduce the volume of the talking substantially.

[Pittylabelle]

Thanks for your reply!

Sure, the sliders are the main adjustment option of the module "Music Rebalance" and there you can pull down the slider "Vocal".

But as I said, the biggest problem in iZotope RX is that the tool recognizes the babble as such very poorly, so I can not select it at all.

[Dan33185]

One of the reasons I hate standing shows anymore is this exact issue. Standing seems to give everyone free license to talk throughout the whole show, whereas seated shows this is rarely an issue. I personally don't understand it.

[Flynn]

you could also try the vocal de-noise function of rx9 or 10. It reduces the chatter a bit or at least to me. You can boost certain things after that. Just a thought.

[Pittylabelle]

One of the reasons I hate standing shows anymore is this exact issue. Standing seems to give everyone free license to talk throughout the whole show, whereas seated shows this is rarely an issue. I personally don't understand it.

I agree with you completely.

On the other hand, you can also have bad luck with a seat determined by the organizer. If there are "taper unfriendly" people in your environment, you can also do nothing. If the people then also have to pee every few minutes, and then ask you loudly for forgiveness, and squeeze past you between the rows of seats, you can also forget the recording.

I experience again and again.

[Pittylabelle]

you could also try the vocal de-noise function of rx9 or 10. It reduces the chatter a bit or at least to me. You can boost certain things after that. Just a thought.

You mean the "Dialogue Isolate" module?

This has given me even worse results than "Music Rebalance".

[Flynn]

you could also try the vocal de-noise function of rx9 or 10. It reduces the chatter a bit or at least to me. You can boost certain things after that. Just a thought.

You mean the "Dialogue Isolate" module?

This has given me even worse results than "Music Rebalance".

Nope. Vocal De-Noise module. Located just below spectral repair

[Pittylabelle]

Nope. Vocal De-Noise module. Located just below spectral repair

What is your exact procedure with the Vocal De-Noise module to filter and reduce the chatter?

[Flynn]

It depends on the recording. Typically for my needs I make sure its set for music and gentle. You can let it be in adaptive mode and do most of the work for you if you are new to the module. Just play with it and see what you like.

here is a tutorial.

https://www.youtube.com/watch?v=csCOPUVmnho

[Pittylabelle]

It depends on the recording. Typically for my needs I make sure its set for music and gentle. You can let it be in adaptive mode and do most of the work for you if you are new to the module. Just play with it and see what you like.

here is a tutorial.

https://www.youtube.com/watch?v=csCOPUVmnho

Tanks for the link. By the way, I had already found the tutorial.

Unfortunately, the tutorial describes how to free non-dynamic, constant room noise from spoken words, which is nearly the opposite of what I want to do. ;-)

As the title of my thread says, isn't it the case that dynamic noise such as chatter can only be effectively filtered out when a kind of AI is used? An AI that has learned what concrete spoken speech is. I think a noiseprint that contains constant noise can't really work here.

[EmRR]

This stuff keeps getting better.  I've a number of stealth recordings in which someone I'm with drunkenly talks to me the whole show, and so far I can't get it out.  But it's coming! 

[Pittylabelle]

If someone is interested, here is my unedited recording with all the chatter:

KRAFTWERK 3-D - 2022-08-28 Bonn Germany 129.38 AUD MC
http://www.dimeadozen.org/torrents-details.php?id=736935

Cheers

Michael

[checht]

Dgital editing is improving. And it can only do so much.

I think in this situation it's about positioning, and being willing to go for a stack tape. Proximity increases signal/noise.

Just go stand directly in front of the speakers haha.

Then you'll still likely have to do some work in post, but ideally it'll result in something you're happy with.

[WiFiJeff]

I'm sorry to say, I've never experienced such an unfocused, nervous and babbling audience as at this concert.

We all look forward to the day when Artificial Intelligence will be able to overcome Natural Stupidity.  Not there yet.

[ballerusk]

Working on one of my recordings now in RX10, trying to remove a chatter over a symphonic tune in the beginning, an intro to the concert. Found the best result from marking the talking and attenuate using Spectral Repair, but this seems cumbersome and will take a long time to go through the whole recording... :/

[roffels]

Working on one of my recordings now in RX10, trying to remove a chatter over a symphonic tune in the beginning, an intro to the concert. Found the best result from marking the talking and attenuate using Spectral Repair, but this seems cumbersome and will take a long time to go through the whole recording... :/

It sure does. Sometimes its almost a blessing when the talking is so pervasive and baked in, I don't bother trying to address it. The erratic stuff is so frustrating.

[checht]

Any thoughts on deleterious impact of using declick on an entire track rather than intermittantly?

Sat next to an avid clapper Mon night, and getting tired of fixing individual spots. Worried it might take out drum hits or somethign...

[capnhook]

Any thoughts on deleterious impact of using declick on an entire track rather than intermittantly?

Sat next to an avid clapper Mon night, and getting tired of fixing individual spots. Worried it might take out drum hits or somethign...

Highlight a short section to declick, select "output clicks only" and listen to the section to see what you're capturing.  If you don't approve, slide the sliders until you get it right, then apply it.

[nassau73]

Any thoughts on deleterious impact of using declick on an entire track rather than intermittantly?

Sat next to an avid clapper Mon night, and getting tired of fixing individual spots. Worried it might take out drum hits or somethign...

I've found that most of the time, declick doesn't affect the drums - even if there's clapping along with the drums.

However, I have run into a few instances where the drums were severely distorted. So, personally, I would stick with working only on the sections with the offensive claps.

It really does take time but it makes a difference. Kinda like with removing whistles. I find that sometimes once I remove the whistles and their harmonics, I listen back and here a sort of "whoosh" sound, so I'll go back and attenuate the "whoosh".

I don't know if any sort of AI would be able to make those kinds of judgements. All the cleanup IS time consuming but worth it for a nicer listening experience.

[checht]

Any thoughts on deleterious impact of using declick on an entire track rather than intermittantly?

Sat next to an avid clapper Mon night, and getting tired of fixing individual spots. Worried it might take out drum hits or somethign...

I've found that most of the time, declick doesn't affect the drums - even if there's clapping along with the drums.

However, I have run into a few instances where the drums were severely distorted. So, personally, I would stick with working only on the sections with the offensive claps.

It really does take time but it makes a difference. Kinda like with removing whistles. I find that sometimes once I remove the whistles and their harmonics, I listen back and here a sort of "whoosh" sound, so I'll go back and attenuate the "whoosh".

I don't know if any sort of AI would be able to make those kinds of judgements. All the cleanup IS time consuming but worth it for a nicer listening experience.

Great points. I've resigned myself to dealing w the clapping sessions individually. Most of 'em are at song ends, so went through and listened to all the ends as a first step. The rest seem more doable now.

Have had same experience using spectral repair w whistles and yells. This stuff takes time, but results are amazing. Last week I released a 2004 Mule show that had some odd repeating diginoise for 15 minutes in the 2nd set, bad enough that I hadn't put the show out. De-click 'periodic clicks' fixed it wonderfully. Amazing to get to share that 18 years later!

[rigpimp]

Any thoughts on deleterious impact of using declick on an entire track rather than intermittantly?

Sat next to an avid clapper Mon night, and getting tired of fixing individual spots. Worried it might take out drum hits or somethign...

Highlight a short section to declick, select "output clicks only" and listen to the section to see what you're capturing.  If you don't approve, slide the sliders until you get it right, then apply it.

But make sure to UNCHECK the Output Clicks Only button before you click Render!  A suggestion from a friend...OK, it was me.  I'm the one that keeps forgetting!   

[nulldogmas]

But make sure to UNCHECK the Output Clicks Only button before you click Render!  A suggestion from a friend...OK, it was me.  I'm the one that keeps forgetting!   

I thought that was only me!

[robgronotte]

Old thread, but just wanted to tell everyone about the wonders of Ultimate Vocal Remover 5.

I had tried to remove unwanted talking with iZotope as nulldogmas explained above, but I could never get good results.  However, this free program has worked very well for me almost every time I have used it.  It hasn't been able to remove some very quiet talking, but it has done a great job removing all the loud talking, and it usually can remove "woo-hoo's" as well.

I have cut out sections with talking, run them through the program, which splits it into instrumental and vocal tracks, then pasted the instrumental track back into the original recording.  Listening to the vocal track is also interesting, you can hear just what the douchebags were talking with their friends about!

The only time I found it didn't work well was when there was talking over the very beginning of a song, when the drummer did a tapping count in (four taps of something in a row, not sure exactly what he was tapping).  Unfortunately the tapping was put into the vocal track instead of the instrumental track.  And of course you can't really use it for sections where people talk while there is singing onstage, or it will take out the singing as well.  Maybe someday that can be perfected as well.

https://ultimatevocalremover.com/

[checht]

Old thread, but just wanted to tell everyone about the wonders of Ultimate Vocal Remover 5.

I had tried to remove unwanted talking with iZotope as nulldogmas explained above, but I could never get good results.  However, this free program has worked very well for me almost every time I have used it.  It hasn't been able to remove some very quiet talking, but it has done a great job removing all the loud talking, and it usually can remove "woo-hoo's" as well.

I have cut out sections with talking, run them through the program, which splits it into instrumental and vocal tracks, then pasted the instrumental track back into the original recording.  Listening to the vocal track is also interesting, you can hear just what the douchebags were talking with their friends about!

The only time I found it didn't work well was when there was talking over the very beginning of a song, when the drummer did a tapping count in (four taps of something in a row, not sure exactly what he was tapping).  Unfortunately the tapping was put into the vocal track instead of the instrumental track.  And of course you can't really use it for sections where people talk while there is singing onstage, or it will take out the singing as well.  Maybe someday that can be perfected as well.

https://ultimatevocalremover.com/

Thanks, Rob, talking is the thing that I can't remove using RX, so am stoked to give this a try.

[robgronotte]

Old thread, but just wanted to tell everyone about the wonders of Ultimate Vocal Remover 5.

I had tried to remove unwanted talking with iZotope as nulldogmas explained above, but I could never get good results.  However, this free program has worked very well for me almost every time I have used it.  It hasn't been able to remove some very quiet talking, but it has done a great job removing all the loud talking, and it usually can remove "woo-hoo's" as well.

I have cut out sections with talking, run them through the program, which splits it into instrumental and vocal tracks, then pasted the instrumental track back into the original recording.  Listening to the vocal track is also interesting, you can hear just what the douchebags were talking with their friends about!

The only time I found it didn't work well was when there was talking over the very beginning of a song, when the drummer did a tapping count in (four taps of something in a row, not sure exactly what he was tapping).  Unfortunately the tapping was put into the vocal track instead of the instrumental track.  And of course you can't really use it for sections where people talk while there is singing onstage, or it will take out the singing as well.  Maybe someday that can be perfected as well.

https://ultimatevocalremover.com/

Thanks, Rob, talking is the thing that I can't remove using RX, so am stoked to give this a try.

It's not perfect, and I still don't really understand the different settings, but it is very good.  If you don't get the results you want, try a different setting.

Maybe if quite a few people start using this program, we could have a thread about it.

[ejl6495]

Old thread, but just wanted to tell everyone about the wonders of Ultimate Vocal Remover 5.

I had tried to remove unwanted talking with iZotope as nulldogmas explained above, but I could never get good results.  However, this free program has worked very well for me almost every time I have used it.  It hasn't been able to remove some very quiet talking, but it has done a great job removing all the loud talking, and it usually can remove "woo-hoo's" as well.

I have cut out sections with talking, run them through the program, which splits it into instrumental and vocal tracks, then pasted the instrumental track back into the original recording.  Listening to the vocal track is also interesting, you can hear just what the douchebags were talking with their friends about!

The only time I found it didn't work well was when there was talking over the very beginning of a song, when the drummer did a tapping count in (four taps of something in a row, not sure exactly what he was tapping).  Unfortunately the tapping was put into the vocal track instead of the instrumental track.  And of course you can't really use it for sections where people talk while there is singing onstage, or it will take out the singing as well.  Maybe someday that can be perfected as well.

https://ultimatevocalremover.com/

Thanks, Rob, talking is the thing that I can't remove using RX, so am stoked to give this a try.

It's not perfect, and I still don't really understand the different settings, but it is very good.  If you don't get the results you want, try a different setting.

Maybe if quite a few people start using this program, we could have a thread about it.

Rob have you had any experience using this with live music that includes vocals? Messing around with it now on the plane home from some shows in Boulder, pretty cool!

[robgronotte]

Old thread, but just wanted to tell everyone about the wonders of Ultimate Vocal Remover 5.

I had tried to remove unwanted talking with iZotope as nulldogmas explained above, but I could never get good results.  However, this free program has worked very well for me almost every time I have used it.  It hasn't been able to remove some very quiet talking, but it has done a great job removing all the loud talking, and it usually can remove "woo-hoo's" as well.

I have cut out sections with talking, run them through the program, which splits it into instrumental and vocal tracks, then pasted the instrumental track back into the original recording.  Listening to the vocal track is also interesting, you can hear just what the douchebags were talking with their friends about!

The only time I found it didn't work well was when there was talking over the very beginning of a song, when the drummer did a tapping count in (four taps of something in a row, not sure exactly what he was tapping).  Unfortunately the tapping was put into the vocal track instead of the instrumental track.  And of course you can't really use it for sections where people talk while there is singing onstage, or it will take out the singing as well.  Maybe someday that can be perfected as well.

https://ultimatevocalremover.com/

Thanks, Rob, talking is the thing that I can't remove using RX, so am stoked to give this a try.

It's not perfect, and I still don't really understand the different settings, but it is very good.  If you don't get the results you want, try a different setting.

Maybe if quite a few people start using this program, we could have a thread about it.

Rob have you had any experience using this with live music that includes vocals? Messing around with it now on the plane home from some shows in Boulder, pretty cool!

Yes, but it can't tell the difference between the singing you want to keep and the audience chat that you want to get rid of.  However, fewer people talk over the singing anyway, it's more often during some quiet instrumental intros to songs.

I cut the song into pieces with another program then run the instrumental sections through UVR5.
It spits out a vocal track and an instrumental one. I listen to the vocal track to make sure it hasn't caught any music I want to keep, then if all is good I patch the instrument only file back to the rest of the song.

[checht]

That workflow sounds hard. How about:

1. run track through magic, split into vocal + everything else
2. listen to vocal sub-track and slience bits you don't want
3. Mix 2 sub-tracks together
4. Profit

[robgronotte]

That workflow sounds hard. How about:

1. run track through magic, split into vocal + everything else
2. listen to vocal sub-track and slience bits you don't want
3. Mix 2 sub-tracks together
4. Profit

I don't know what 'magic' is, but that sounds worse to me. UVS5 is very slow for long music files, and listening to an hour or longer of only the vocals of a concert recording sounds like torture.

[ballerusk]

I would assume you could listen to the concert while working on other aspects of your recording and make a note where there are unwanted bits so when working in UVS5 you can just jump to those spots.

[ballerusk]

I tried using UVR5 on an old recording and found the "best" result using the following settings:

Choose process method: Demucs
Choose stems: Vocals
Segment: default
Choose demucs model: v4 htdemucs

The result wasn't perfect, but I guess it varies depending on what you feed the program. UVR5 managed to separate some chatter, but most of the "background chatter" seemed to be integrated in the instrumental-file so for me it worked best with clear speech. It also made it a little easier to "see" the talking when viewed in Izotope RX. Not sure how much I will use this going forward, unless there is very obvious talking.

Processed a 1h7m file with the settings above, it took about 1h38m (1.47x realtime) from start to finish on my 2019 iMac (3GHz 6-core i5 40GB DDR4 RAM).

[robgronotte]

It seems to be very good at removing clear talking. Very quiet talking is hit or miss, though generally an improvement.

I have found demucs and MDX give very similar results. The other choice (VR Architecture) seems to work differently, so some talk is better removed with that one, and some better with Demucs or MDX. You may want to try running a passage through both filters to get more removed.