Become a Site Supporter and Never see Ads again!

Author Topic: A.I. audio stem software  (Read 7561 times)

0 Members and 1 Guest are viewing this topic.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
A.I. audio stem software
« on: November 20, 2024, 09:44:47 AM »
 A.I. audio stem software.

There are several audio software's that take a audio wav file and create 4 stems - Vocals, Bass, Percussion and Other instruments such as Izotope RX Music rebalance and Ultimate Vocal Removal Tool.
Spectralayers  takes a audio wav file and create 7 stems - Vocals, Bass, Percussion, brass, piano, guitar and Other instruments.

This thread is for all A.I. audio stem software.

==========================================================================

Izotope RX music rebalance. (RX MR)
https://www.izotope.com/en/products/rx/features/music-rebalance.html - You need to buy RX 11 standard which costs $399 now on sale for $200 at the sweetwater.

Sweetwater has some upgrade versions on sale.
https://www.sweetwater.com/store/manufacturer/iZotope?_queryID=fd9b447736fa472739fea35254b5afe4&_index=production_products

==========================================================================

Ultimate Vocal removal tool. (UVRT)

https://ultimatevocalremover.com/ - UVRT is Free though really needs a high end graphic card with lots of ram to run fast.

==========================================================================

Spectralayers (SL)
https://www.steinberg.net/spectralayers/ - Regular cost $299 onsale 11/20/2024 for $179.

SL has many more unmix options then RX or UVRT such as "unmix components" - Take drum layer and create more stems or bass drum, snare and cymbals. "Unmix crowd noise" - takes out crowd noise from a recording. You can create layers in SL and even take from another layer and add it to the layer you are working in. Very powerful.

Utube video of Beat Deconstruction Unmixing: (Video shows some of the power of SL)
https://www.youtube.com/watch?v=7-VIrWzbJkE

============================================================================

From Music Radar: "iZotope RX 11 vs Steinberg SpectraLayers 11: which is the best spectral editor?"
https://www.musicradar.com/news/izotope-rx-11-vs-steinberg-spectralayers-11

============================================================================
Background:

Have used these stem software's for about a year now. Was mainly using RX music rebalance
as a tool to make a matrix recording of both aud mics and SBD feed. Would split the SBD feed
into Vocals, Bass, Percussion and Other instrument stems. Load SBD feed then audio mics then
each stem into a audio editor. You have 6 tracks to work with now instead of two. If vocals are
low in aud mics then can use the vocal SBD stem and increase the vocals volume. Or if bass is
not there you can use bass SBD stem and increase the volume. The possibilities here are
almost endless. Almost have to change the way you think about processing your audio files.

The RX MR software seems to split better with a SBD feed then splitting the audio mic source. It does
not do a bad job on audio mic source though it is not as clean as the SBD feed stems.

============================================================================

Does it work?

It sure does. Is it perfect - no. Though with vocals and bass seem to come out the best in RX MR
and drums can come out well though there is a bleed with some of the other instruments into the
drum stem. It the "Other instruments" stem where all the rest of the music is. If you have
guitar you want to increase there is no separate stem for that and you have to play with the
 "Other instruments" stem to get it to come out. Some people think there is a metallic sound
from the vocal stem in RX MR.

Ultimate Vocal removal tool (UVRT ) separates the vocals better then RX MR. There are model in
UVRT that will split the audio file into 4 stems. You can even separate vocals into lead vocals
and backup vocals. One disadvantage of UVRT  is it outputs 16 bit audio stems, not able to select
24 bit audio stem files. In RX MR you can get 24 bit audio stem files.

============================================================================

What kind of PC do you need to run these software?

A high end PC.

To process one 60 minute sbd feed with RX MR on my old windows 7 daw took 1 hour 17 minutes.
3-4 years ago bought a laptop windows 11 I7 with lots of ram and fast CPU with built in graphic card
with 2 GB ram took the 60 minutes SBD feed to 18 minutes. Just bought a Win 11 I7 32 GB ram and very
fast CPU with good graphic card with 8 GB memory. Now to process 60 minutes SBD feed to 8 minutes for RM MR.

If record 2 70 minute sets have 4 audio files per source. So have to run each of the 4 SBD files
with RX MR to get stems for the whole show. Now to process all 4 SBD files with my new PC is
less then 30 minutes. Have to do this before I master the show. This causes more work and processing.

With UVRT you really need a graphics card with lots of ram or the software runs very slowly. Can
process in UVRT a 60 minute sbd feed with new pc in 8 minutes though it is only 16 bit audio files
as mentioned above.

SL - unmix song - 6 layers: piano, bass, guitar, drum, vocal and other.
20 min song 24/96 wav file  - 6 stems  - 35 minutes processing time.

============================================================================

Where does this leave us with this new software?

If this software can really split the file into 4 or 7 stems then possibilities are endless. You could even take old SBD and add whatever is missing. This software will get better over time too.

Does anyone else have experience with this or like to chime in about it?

===================================================================

Link to article about different stem tools:
https://www.attackmagazine.com/reviews/the-best/four-of-the-best-stem-separation-tools/ 
« Last Edit: November 20, 2024, 04:09:53 PM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #1 on: November 20, 2024, 09:44:57 AM »
Some observations about SL Benchmarks and capabilities .
===================================================================

unmix song - 6 layers: piano, bass, guitar, drum, vocal and other.

20 min song 24/96 wav file - 6 stems 2 GB 24 bit each - 35 minutes to process

Piano layer - works well for electric not sure about acoustic
Bass Layer - works well full sounding layer with good mix from SBD feed
Guitar Layer - is there though seems to be a little cut off at certain frequency
Drums layer - kick, bass and hy-hat mostly covered. Cymbals cut off at very high end in other instrument layer
Other layer - weird artifacts from the other layers.
Voice layer - works well maybe some cut off freq from other layers.

The "other layer" contains some data from other layers and would be needed for the song to sound complete.

======================================================================

Unmix components:

60 min drum track 24/96 wav file - 50 minute to process

Creates 3 layers - tonal, transient and noise

kick Drum layer
tonal layer- snare drum
transient layer - cymbals

The bass drum layer picks up most of it though it is not completely clean as just kick drum. Snare layer
 is done pretty well though has artifacts from other layers.  Cymbals layer does a good job. Then you can
even use "Unmix componets" and separate the cymbals to hi-hat and rest of cymbals in another layer.

=====================================================================

unmix crowd noise:

60 min SBD set 24/96 wav for unmix crowd noise to process 1h19m.

The unmix crowd noise did work though does pick up some of the instruments in the
crowd layer. In between songs it did a good job and took out most if not all
of the crowd noise. Some screams and clapping was taking out though it was
with the music too. A hand drum was mistaken as crowd noise.  Not sure this works
well enough to be effective without doing editing to put back in music or take out scream/whistles.

====================================================================

unmix chorus:

60 min SBD 24/96 wav set for unmix chorus to process 53m.

Took a previous vocal stem with lead singer and harmony by the band and
ran it with unmix chorus. Output layers lead and backup were created.
Lead vocal came out pretty well though sometimes it would get confused
with backup and lead and it would bleed through. Not sure this works well
enough to be effective without doing more editing to add whatever is missing
to another layer or the layer you are working in.
====================================================================

unmix song

9 min song 16 44 wav fareed - 2 acoustic guitar -  5 min to process

Only did 2 layers guitar and other. The guitar layer did not separate the 2
acoustic guitars and the other layer contained a fair amount of guitars when
they both guitars were playing loud.

=========

unmix song

19 min song 16 44 wav flex band - 3 piece jazz combo - piano, bass and drums stems - took 20 minute to process.

Outputted 4 layers - other, piano, bass and drums. Bass layer had almost nothing
in it music wise. Drums were pretty well represented though high end seemed to be
cut off. Acoustic piano was well done though would bleed into the "others layer'.
Other layer contained music bleed from piano and drums.

===============================================================================
spectralayers 11 commands

open wav file

module > unmix song

file > Export > Layers (creates .wav files for each layer open for the unmix options then import layers into DAW)
« Last Edit: November 20, 2024, 11:34:26 AM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline mccordo

  • Trade Count: (14)
  • Taperssection Member
  • ***
  • Posts: 674
  • Gender: Male
  • Area Man
Re: A.I. audio stem software
« Reply #2 on: November 20, 2024, 11:05:25 AM »
Thanks for posting this. I've been considering a software upgrade and this was just the type of info I needed to help make my decision. Looks like Izotope RX is the answer for me.
Mics: DPA 2012 Cards, DPA 4015c Wide Cards, DPA 4061-CORE Omnis, AKG ck63 Hypercards, 1x AKG nBob Actives, MJE-384K Roadster (Michael Joly modded caps), Audix M1280 Hypercards
Pres: Grace Design Lunatec V2, SoundDevices MixPre, Edirol UA-5, Church Audio CA-9200, Naiant PIPsqueak
Recorders: SoundDevices MixPre-6, SoundDevices MixPre-3, 2x Tascam DR-100mkII, Zoom F3, Sony PCM-M10, Sony PCM-A10, Deity PR-2

Offline checht

  • Site Supporter
  • Trade Count: (7)
  • Taperssection Member
  • *
  • Posts: 855
  • Let's meet at alternate foods at the break
Re: A.I. audio stem software
« Reply #3 on: December 04, 2024, 03:45:56 PM »
OK, moved my post from another thread here:

I separate out vocals on most every recording. For SBD vocal feeds, it's a great way to debleed.
For aud 2 track recordings, I separate out vocals and mix them back in in parallel to add presences.
Really cures that 'distant' sound that plagues aud recordings. I find it especially helpful on recordings made with Neumann kmi84s from back in the day.

I've been using RX, currently on v9. Today I tried ultimate vocal remover and I'm shocked by how much better it works. Output is much cleaner intem of not clcutting off the beginning of phrases, and not including sax or guitar that would make it through rx. Also, when listening to it solo, the uvr track is drastically more musicall and natural sounding. To me, RX has always sounded kinda alien on its own.

uvr runs much faster than rx on my mac mini m3. And it's free.

What do others use stem separation for, what softwar do you use, and what are your thoughts?
Anyone else compare outputs?

Just uploaded samples to dropbox. There's the original sbd vocals feed, full of stage bleed. Then 2 configs of uvr, then rx music rebalance.
Thoughts?
:
https://www.dropbox.com/scl/fo/izqcnyfgo1kmjc4utaru7/ADXknEXvzjmA0vVEmr4OjLc?rlkey=z1j830ojjrxkecnevykzb2c5i&dl=0
MK41s, MK22s; Vanguard V1s matched pair
Schoeps kcy5, nbob actives
Naiant PFA 60v, PFA 48v, IPA
Sound Devices MP-6II; Sony PCM-A10

Recordings at LMA

Offline nulldogmas

  • Trade Count: (6)
  • Taperssection All-Star
  • ****
  • Posts: 1843
    • How I Escaped My Uncertain Fate
Re: A.I. audio stem software
« Reply #4 on: December 04, 2024, 06:02:55 PM »
What do others use stem separation for, what softwar do you use, and what are your thoughts?

I use stem separation mostly to rebalance different instruments/vocals, or very occasionally to apply EQ to one but not the others. (Say, if a kick drum is too loud but I don't want to reduce the bass guitar in the same frequency range.) I don't use it on most recordings, but maybe 20-30% of them?

I've tried both RX and UVR and agree that UVR does a better job isolating vocals, though they're both very good for most uses. (I use the MDX23C-InstVoc HQ setting, at Rob G's suggestion.) I haven't tried UVR for multi-stem separation yet — still waiting to hear reports on which settings people find work best.

In either case, I usually export the stems and then remix them in Audacity, where it's easier to play with the sliders on the fly.

Offline jefflester

  • Trade Count: (2)
  • Taperssection All-Star
  • ****
  • Posts: 1703
  • Gender: Male
Re: A.I. audio stem software
« Reply #5 on: December 04, 2024, 06:28:50 PM »
I did a show this weekend with my band, individual instruments > F8, only output available from the board was an FX send and I got a lot more piano and acoustic guitars than vocals so I want to try UVR to pull out just the vocals. What should I use for "Overlap"? DLing the "MDX23C-InstVoc HQ" just now.
DPA4061 HEB -> R-09 / AT943 -> CA-UGLY -> R-09
AKG CK63 -> nBob actives -> Baby NBox -> R-09/DR2d
AKG CK63 -> AKG C460B -> Zoom F8/DR-680MKII
Line Audio CM4/Superlux S502/Samson C02/iSK Little Gem/Sennheiser E609/Shure SM57 -> Zoom F8/DR-680MKII (multitracked band recordings)

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #6 on: December 05, 2024, 04:08:48 AM »
I did a show this weekend with my band, individual instruments > F8, only output available from the board was an FX send and I got a lot more piano and acoustic guitars than vocals so I want to try UVR to pull out just the vocals. What should I use for "Overlap"? DLing the "MDX23C-InstVoc HQ" just now.

I use UVR5 often, mostly to remove crowd noise from instrumental portions of songs.  I clip out the portion I want to clean up, run it through UVR5, and then patch back in the clean option.  As noted above, the best one I have found is "MDX23C-InstVoc HQ", which has to be downloaded separately (also free and very easy).
I don't know what the "Segment Size" and "Overlap" options refer to, so I have just left them at the default, which was 256 and 8.  If anyone knows more about options on UVR5 I would love to understand it better.

Actually I almost always run in "Ensemble Mode" [with Max Spec/Min Spec setting] which runs the file through several different filters at the same time.  In addition to the one mentioned above, I use "VR Arc1_HP-UVR" and "Demucs v4|htdemucs" (both included in main download).  It barely takes any longer than using the MDX23C alone, as that one is very slow compared with most of the algorithms.   I get 4 outputs - one for each of the 3 processes, plus a combined version.  If the MDX23C version doesn't sound perfect to me, I will listen to the others and possibly use one of those instead.

If anyone else wants to use UVR5 for similar results, feel free to ask any questions and I could give some more detailed info about what I do and the results I get.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #7 on: December 05, 2024, 06:07:41 PM »
Was not sure if this thread would gain any traction so have not posted any additional info. No expert in this just posting obervations/thoughts. Will not say anything about the below files till other listen and give their opinion.
(Links good for a week)

==================================================

Samples:

1) 2 song - 24/96 sbd wav files and created 4 stems - vocal, bass, drum and "other instruments" in RX, UVR and Steinberg Spectral Layers (SL).

2) Linked original 24/96 wav file.

3) Created 6 stems of the same original file in SL (6 stems Vocal, bass, drum, other, piano and guitar) for others to compare how it splits the piano and guitar into separate stems.

Did no processing except take the 16 bit UVR and created 24/96 files so could compare them in my daw. Created flacs from the wavs to conserve file space.   

==================================================

 2 songs - 24/96 sbd wav files and created 4 stems - vocal, bass, drum and "other instruments" in RX, UVR and Steinberg Spectral Layers (SL)

Original 24/96 wav file -
https://www.transfernow.net/dl/20241205Of8Qbegm/A5Cyax6n

RX - 4 stems - vocal, bass, drum and "other instruments" -3 min 9 seconds to create 4 stems
https://we.tl/t-J0a6citpHC

UVR- 4 stems - vocal, bass, drum and "other instruments - 2 min 52 seconds to create 4 stems (Used "htdemucs" model to process the 4 stems)
https://we.tl/t-iZ7LAC1zi1

SL - 4 stems -  vocal, bass, drum and "other instruments - 8 minutes to create 4 stems
https://we.tl/t-zP1ihW2Llb

==================================================

SL - 6 stems Vocal, bass, drum, other, piano and guitar. - 8 minutes to create 6 stems
https://www.transfernow.net/dl/20241205Bb9h9fvL/gxaQu3kr

===========================================

Any comments?
« Last Edit: December 06, 2024, 11:18:06 AM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #8 on: December 05, 2024, 06:58:18 PM »
Phil, have you tried the Spectralayers function of separation of different vocals?

I tried it once but didn't understand how I was supposed to "train" it to learn the main vocalist.
Was hoping it could cut out audience chat over the singing.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #9 on: December 06, 2024, 06:35:40 AM »
Phil, have you tried the Spectralayers function of separation of different vocals?

I tried it once but didn't understand how I was supposed to "train" it to learn the main vocalist.
Was hoping it could cut out audience chat over the singing.
If I understand what you want to do correctly...then suggest doing the following procedure in SL.

=====================================================================

Create vocal stem and open it in SL.

Select "unmix Multiple Voices" in the modules.

Use cursor and Highlight a 20 second piece or so for main vocalist in the vocal stem
Then in the  "unmix Multiple Voices" window Click "Register Voice" - In SL it creates "voice 1"  in the "unmix Multiple Voices" window.

Now you can select the second vocalist and do the same as above and the then this will create a "Voice 2" for the second vocalist.

Now click "apply" in the  "unmix Multiple Voices" window and SL creates layers for "Voice 1", "Voice 2", "Non_voice" and  "Non-Un-mixed".

File > Export > Layers

=======================================================================
Hope you can understand what I wrote.

=======================================================================

SpectraLayers 11 vs Ultimate Vocal Remover:
https://www.youtube.com/watch?v=u3Ntpdt-pCU
« Last Edit: December 06, 2024, 11:23:37 AM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #10 on: December 06, 2024, 12:58:11 PM »
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #11 on: December 06, 2024, 01:06:18 PM »
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #12 on: December 06, 2024, 02:26:54 PM »
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.

How does that work?

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #13 on: December 07, 2024, 07:18:09 AM »
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.

How does that work?
From what I have seen so far:

unmix Multiple Voices - is to separate 2 or more vocalist to have one vocal layer for each singer.

Unmix chorus -  Output layers lead and backup are created. Seems this would be good choice if had one main singer and then harmony with rest of band.

===================================================

unmix chorus:

60 min SBD 24/96 wav set for unmix chorus to process 53m.

Took a previous vocal stem with lead singer and harmony by the band and
ran it with unmix chorus. Output layers lead and backup were created.

Lead vocal came out pretty well though sometimes it would get confused
with backup and lead and it would bleed through.

=======================================================

The wetransfer links above end tomorrow. Hope someone DL'ed them and has some comments. Gives a good idea of what this software can do.

« Last Edit: December 07, 2024, 07:26:14 AM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline ballerusk

  • Trade Count: (3)
  • Taperssection Member
  • ***
  • Posts: 257
  • Soft spot for the sweet spot
Re: A.I. audio stem software
« Reply #14 on: December 18, 2024, 04:06:15 PM »
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.
Schoeps MK41s > Schoeps CMRs > Naiant Tinybox > Sony PCM-M10

Offline AbbyTaper

  • Trade Count: (0)
  • Taperssection Regular
  • **
  • Posts: 241
Re: A.I. audio stem software
« Reply #15 on: December 18, 2024, 04:23:10 PM »
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.

I tried it once, and would describe the results as "mixed".

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #16 on: December 18, 2024, 10:55:21 PM »
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.

I tried it once, and would describe the results as "mixed".

I have found the whole program very difficult to use. Very disappointing, because the features seem great.

Offline lavaux

  • Trade Count: (0)
  • Taperssection Newbie
  • *
  • Posts: 4
Re: A.I. audio stem software
« Reply #17 on: December 23, 2024, 09:10:31 PM »
Based on quick research, seems like SpectralLayers was built off of Spleeter, which is the old method of extracting elements. They now use Demucs, which you get for free with UVR (but UVR doesn't have the vocal learning feature that Spectralayers has.

I think MVSEP and X-Minus are the most cutting edge, with UVR right behind them. I use UVR to remove audience, then load it up into Izotope RX and spectrally remove all the talking/coughing/non-applause.

Here's a really great research document that showcases all the current updates and movements in the AI audio separation field:
https://docs.google.com/document/d/17fjNvJzj8ZGSer7c7OFe_CNfUKbAxEh_OBv94ZdRG5c/edit?usp=sharing

Offline if_then_else

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 497
Re: A.I. audio stem software
« Reply #18 on: December 24, 2024, 03:40:02 AM »
There are a few other options that haven't been mentioned in this thread yet.

  • Remix from Acon Digital (https://acondigital.com/products/remix) that's also part of Acoustica Premium Edition. I've got the latter and the tool works great for balancing instruments in a mix. But it wouldn't be my top choice for stem separation.
  • Rip X (https://hitnmix.com) which - according to some reviews by sound and mixing engineers - is supposedly the most advanced tool of them all. That said, the DAW itself in which stem separation is integrated looks like it could need improving.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #19 on: December 24, 2024, 06:59:03 AM »
Have used RX, UVR and SL using many different files and genres. Used the Demus in UVR and found that SL and UVR sounded the best in stem separation and almost similar. RX sometimes has this metallic sound to it. Though found mixed results depending on the genre I used with each software. With RX and UVR they just split stems and that is it. No editing features. SL has these editing features like RX with the addition of the eraser tool, batch processing, select similar and ability to add layers or extract from layers and add to another layer or create a new layer. This is a big step IMO.

Have a fast PC - time to process 70 minute set in following software with 4 stem extraction:
RX - 8 minutes
UVR - 8 minutes
SL 60-70 minutes - you can get around the slow processing time by using the batch processing tool and run it over night.

Tried this over the weekend took a audience recording with crowd noise in the back ground and ran "unmix crowd noise" in SL. Was surprised  how well it took out the chatter from the floor noise. What was left after extracting the crowd noise almost sounded like a SBD though some vocals were taken out with the crowd noise. Then selected the vocals manually in a short section in the crowd noise layer and  ran "select similar". It ran for hours to try and find all of them. Did an ok job. If you extracted just the vocal then ran "unmix crowd noise" it might make a nice sounding audio file. SL is like photoshop and can do many more features then just splitting the files. Though as robgronotte suggests its not as easy program to use and would take time to figure out how to really use it.

====================================================

Thanks to  lavaux and  if_then_else for their input and comments. It is appreciated.

Will look into the software you guys posted about. Thank you!
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #20 on: January 10, 2025, 09:52:25 PM »
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #21 on: January 12, 2025, 11:07:08 AM »
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.
Good job figuring out how to use the tools. Cool that you used SL then UVR5 then SL.

That is my thought pattern too. Will have to use the tools together to get what you want. RX and UVR5 are good at separating a file into stems though that is all they do. SL does so much more though you have to use their tools then manually edit the output to separate what you want in the file and (Noise, chatter, instruments) that you want to take out.

Tried "unmix crowd noise" and thought it did a good job. It completely took out chatter between songs. Though in some songs it took out some of the vocals that the program thought was crowd noise. Wish there was a way to only select different sections of the file then run the tool.



 
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline AbbyTaper

  • Trade Count: (0)
  • Taperssection Regular
  • **
  • Posts: 241
Re: A.I. audio stem software
« Reply #22 on: February 01, 2025, 04:37:34 PM »
Has anyone tried Moises?  Not free, but under $3 per month on an annual contract.

Offline robgronotte

  • Trade Count: (0)
  • Taperssection Member
  • ***
  • Posts: 371
Re: A.I. audio stem software
« Reply #23 on: February 01, 2025, 07:26:44 PM »
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.
Good job figuring out how to use the tools. Cool that you used SL then UVR5 then SL.

That is my thought pattern too. Will have to use the tools together to get what you want. RX and UVR5 are good at separating a file into stems though that is all they do. SL does so much more though you have to use their tools then manually edit the output to separate what you want in the file and (Noise, chatter, instruments) that you want to take out.

Tried "unmix crowd noise" and thought it did a good job. It completely took out chatter between songs. Though in some songs it took out some of the vocals that the program thought was crowd noise. Wish there was a way to only select different sections of the file then run the tool.

Seems like there should be a way to only select part of a file but I haven't found it. I cut the recording into pieces with another program and then run through just the part I want to use it on. Same thing for UVR5.

Offline bmubart

  • Trade Count: (0)
  • Taperssection Newbie
  • *
  • Posts: 8
Re: A.I. audio stem software
« Reply #24 on: February 04, 2025, 06:15:12 AM »
This is my workflow for noisy crowds (for what it is worth):
- Spectralayers: Unmix Song in Voice + Other (= 2 layers)
- Unmix the Voice further with Unmix Crowd.  You become 2 new layers Foreground + Crowd

You can now re-build the whole track in your DAW (Reaper...) using the layers:
1) Other
2) Foreground
3) Crowd

Listen to the result. 
You can now edit the Crowd-layer on the fly in eg Izotope RX or a wave editor to eliminate crowd noise.
Render
--> same recording with almost no crowd

This is very time consuming but the result is very pleasing (imho).
This doesn't work well when there are singers in the crowd which voice blend with the lead-singer

All unmixing in Spectralayers can be done in batch




« Last Edit: February 04, 2025, 06:26:49 AM by bmubart »

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #25 on: February 06, 2025, 06:36:38 AM »
Good work  bmubart. Thanks for sharing.

As TS members figure out how to use these software's  then maybe they could share their processes so others can learn from them.

SL is an advanced software and getting it to do what you want will take practice and time.
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline bmubart

  • Trade Count: (0)
  • Taperssection Newbie
  • *
  • Posts: 8
Re: A.I. audio stem software
« Reply #26 on: February 06, 2025, 09:28:03 AM »
Good work  bmubart. Thanks for sharing.

As TS members figure out how to use these software's  then maybe they could share their processes so others can learn from them.

SL is an advanced software and getting it to do what you want will take practice and time.

Thanks!

I'm a new user to SL too but it is really not that hard at all for the things I need it: making layers/stems.

I do these steps in batch mode (overnight):
- select the tracks I want to unmix
- select Unmix Song and Seperate them in Layers.  You can choose how many layers (drum, bass, guitar, vocals...) but mostly 'voice' and 'others' will do so each track becomes two wave-files
- select the folder where the batch job has to export the layers (= wave files)

Then: go to the folder with the unmixed layers (= wave file per stem)
- in batch (again):
- choose the voice-layers
- choose 'Unmix Crowd' and select the folder to export the voice layers: Crowd + Foreground

After that you will become all the stems with the voice seperated in two layers: crowd + foreground
Take these stems to rebuild the full track in your DAW:

- others
- foreground
- crowd

(so don't use the original voice-layer)

Play this 3 together in your DAW (= full track WITH crowd-noise)
Eliminate the crowd noise by playing with volume envelopes or by reducing it / deleting it in your wave editor (I use Izotope RX)
Listen again = full track with less crowd noise

Hope this helps?



Offline checht

  • Site Supporter
  • Trade Count: (7)
  • Taperssection Member
  • *
  • Posts: 855
  • Let's meet at alternate foods at the break
Re: A.I. audio stem software
« Reply #27 on: February 06, 2025, 11:15:32 AM »
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?
MK41s, MK22s; Vanguard V1s matched pair
Schoeps kcy5, nbob actives
Naiant PFA 60v, PFA 48v, IPA
Sound Devices MP-6II; Sony PCM-A10

Recordings at LMA

Offline nulldogmas

  • Trade Count: (6)
  • Taperssection All-Star
  • ****
  • Posts: 1843
    • How I Escaped My Uncertain Fate
Re: A.I. audio stem software
« Reply #28 on: February 06, 2025, 03:54:17 PM »
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?

I'm on a Mac, but haven't tried ensemble yet. What stems are you choosing?

Offline phil_er_up

  • Trade Count: (9)
  • Taperssection All-Star
  • ****
  • Posts: 1326
Re: A.I. audio stem software
« Reply #29 on: February 08, 2025, 12:22:27 PM »
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?
Hi Chris,

On old windows PC without a graphics card it would take hours to run UVR with 4 stems and a 2 GB wav file and sometimes would hang my pc.
On new fast win 11 pc with graphics card and 8 gb memory takes only 8 minutes to process 4 stems with 2 GB wav file.

Believe you told me it was only 8 minutes to process one stem on your mini-mac thought that was really fast though was surprised due to you did not have a graphic card.
Have looked at task manager in win 11 when UVR is running and my graphics card is running at 100%. Though it does eat memory too. If creating 4 stems with a 2 GB file you need 4-2gb files output that is 8 Gb in file output from UVR.

So assumed you needed a graphic card for UVR to run fast and not crash your machine.
« Last Edit: February 08, 2025, 12:29:42 PM by phil_er_up »
Everyday is a gift. Enjoy each one!
Forward motion bring positive results.

Offline checht

  • Site Supporter
  • Trade Count: (7)
  • Taperssection Member
  • *
  • Posts: 855
  • Let's meet at alternate foods at the break
Re: A.I. audio stem software
« Reply #30 on: February 09, 2025, 01:15:42 PM »
Hiya Pat,

Except for the Mac Pro, Macs don't use graphics. cards these days.

I'm on the road w Los Lobos, so using Macboook Air and just going with no ensemble, just MXR Main. 1.25 hour show takes around 45 min to process. In this application, I'm recording omt4 with mk41s x/y PAS in the center, and mk22s 4' spread. I'm not too picky about extraction in this case, as I'm pulling the vocals from the 41s to mix back in in parallel with the 2 stereo tracks, a variety of vocal sweetening.

Here's a track from Thursday night:
https://archive.org/details/LosLobos2025-01-06/ll20250206.OMT4.2448-02.flac

This is my current state of the art, combining AI extraction with omt4 recording. Fun times.

Not very good picture attached showing omt4 boom at the triple door in Seattle.
« Last Edit: February 09, 2025, 01:17:47 PM by checht »
MK41s, MK22s; Vanguard V1s matched pair
Schoeps kcy5, nbob actives
Naiant PFA 60v, PFA 48v, IPA
Sound Devices MP-6II; Sony PCM-A10

Recordings at LMA

 

RSS | Mobile
Page created in 0.075 seconds with 55 queries.
© 2002-2025 Taperssection.com
Powered by SMF