Taperssection.com

Gear / Technical Help => Post-Processing, Computer / Streaming / Internet Devices & Related Activity => Topic started by: phil_er_up on November 20, 2024, 09:44:47 AM

Title: A.I. audio stem software
Post by: phil_er_up on November 20, 2024, 09:44:47 AM
 A.I. audio stem software.

There are several audio software's that take a audio wav file and create 4 stems - Vocals, Bass, Percussion and Other instruments such as Izotope RX Music rebalance and Ultimate Vocal Removal Tool.
Spectralayers  takes a audio wav file and create 7 stems - Vocals, Bass, Percussion, brass, piano, guitar and Other instruments.

This thread is for all A.I. audio stem software.

==========================================================================

Izotope RX music rebalance. (RX MR)
https://www.izotope.com/en/products/rx/features/music-rebalance.html - You need to buy RX 11 standard which costs $399 now on sale for $200 at the sweetwater.

Sweetwater has some upgrade versions on sale.
https://www.sweetwater.com/store/manufacturer/iZotope?_queryID=fd9b447736fa472739fea35254b5afe4&_index=production_products

==========================================================================

Ultimate Vocal removal tool. (UVRT)

https://ultimatevocalremover.com/ - UVRT is Free though really needs a high end graphic card with lots of ram to run fast.

==========================================================================

Spectralayers (SL)
https://www.steinberg.net/spectralayers/ - Regular cost $299 onsale 11/20/2024 for $179.

SL has many more unmix options then RX or UVRT such as "unmix components" - Take drum layer and create more stems or bass drum, snare and cymbals. "Unmix crowd noise" - takes out crowd noise from a recording. You can create layers in SL and even take from another layer and add it to the layer you are working in. Very powerful.

Utube video of Beat Deconstruction Unmixing: (Video shows some of the power of SL)
https://www.youtube.com/watch?v=7-VIrWzbJkE

============================================================================

From Music Radar: "iZotope RX 11 vs Steinberg SpectraLayers 11: which is the best spectral editor?"
https://www.musicradar.com/news/izotope-rx-11-vs-steinberg-spectralayers-11

============================================================================
Background:

Have used these stem software's for about a year now. Was mainly using RX music rebalance
as a tool to make a matrix recording of both aud mics and SBD feed. Would split the SBD feed
into Vocals, Bass, Percussion and Other instrument stems. Load SBD feed then audio mics then
each stem into a audio editor. You have 6 tracks to work with now instead of two. If vocals are
low in aud mics then can use the vocal SBD stem and increase the vocals volume. Or if bass is
not there you can use bass SBD stem and increase the volume. The possibilities here are
almost endless. Almost have to change the way you think about processing your audio files.

The RX MR software seems to split better with a SBD feed then splitting the audio mic source. It does
not do a bad job on audio mic source though it is not as clean as the SBD feed stems.

============================================================================

Does it work?

It sure does. Is it perfect - no. Though with vocals and bass seem to come out the best in RX MR
and drums can come out well though there is a bleed with some of the other instruments into the
drum stem. It the "Other instruments" stem where all the rest of the music is. If you have
guitar you want to increase there is no separate stem for that and you have to play with the
 "Other instruments" stem to get it to come out. Some people think there is a metallic sound
from the vocal stem in RX MR.

Ultimate Vocal removal tool (UVRT ) separates the vocals better then RX MR. There are model in
UVRT that will split the audio file into 4 stems. You can even separate vocals into lead vocals
and backup vocals. One disadvantage of UVRT  is it outputs 16 bit audio stems, not able to select
24 bit audio stem files. In RX MR you can get 24 bit audio stem files.

============================================================================

What kind of PC do you need to run these software?

A high end PC.

To process one 60 minute sbd feed with RX MR on my old windows 7 daw took 1 hour 17 minutes.
3-4 years ago bought a laptop windows 11 I7 with lots of ram and fast CPU with built in graphic card
with 2 GB ram took the 60 minutes SBD feed to 18 minutes. Just bought a Win 11 I7 32 GB ram and very
fast CPU with good graphic card with 8 GB memory. Now to process 60 minutes SBD feed to 8 minutes for RM MR.

If record 2 70 minute sets have 4 audio files per source. So have to run each of the 4 SBD files
with RX MR to get stems for the whole show. Now to process all 4 SBD files with my new PC is
less then 30 minutes. Have to do this before I master the show. This causes more work and processing.

With UVRT you really need a graphics card with lots of ram or the software runs very slowly. Can
process in UVRT a 60 minute sbd feed with new pc in 8 minutes though it is only 16 bit audio files
as mentioned above.

SL - unmix song - 6 layers: piano, bass, guitar, drum, vocal and other.
20 min song 24/96 wav file  - 6 stems  - 35 minutes processing time.

============================================================================

Where does this leave us with this new software?

If this software can really split the file into 4 or 7 stems then possibilities are endless. You could even take old SBD and add whatever is missing. This software will get better over time too.

Does anyone else have experience with this or like to chime in about it?

===================================================================

Link to article about different stem tools:
https://www.attackmagazine.com/reviews/the-best/four-of-the-best-stem-separation-tools/ 
Title: Re: A.I. audio stem software
Post by: phil_er_up on November 20, 2024, 09:44:57 AM
Some observations about SL Benchmarks and capabilities .
===================================================================

unmix song - 6 layers: piano, bass, guitar, drum, vocal and other.

20 min song 24/96 wav file - 6 stems 2 GB 24 bit each - 35 minutes to process

Piano layer - works well for electric not sure about acoustic
Bass Layer - works well full sounding layer with good mix from SBD feed
Guitar Layer - is there though seems to be a little cut off at certain frequency
Drums layer - kick, bass and hy-hat mostly covered. Cymbals cut off at very high end in other instrument layer
Other layer - weird artifacts from the other layers.
Voice layer - works well maybe some cut off freq from other layers.

The "other layer" contains some data from other layers and would be needed for the song to sound complete.

======================================================================

Unmix components:

60 min drum track 24/96 wav file - 50 minute to process

Creates 3 layers - tonal, transient and noise

kick Drum layer
tonal layer- snare drum
transient layer - cymbals

The bass drum layer picks up most of it though it is not completely clean as just kick drum. Snare layer
 is done pretty well though has artifacts from other layers.  Cymbals layer does a good job. Then you can
even use "Unmix componets" and separate the cymbals to hi-hat and rest of cymbals in another layer.

=====================================================================

unmix crowd noise:

60 min SBD set 24/96 wav for unmix crowd noise to process 1h19m.

The unmix crowd noise did work though does pick up some of the instruments in the
crowd layer. In between songs it did a good job and took out most if not all
of the crowd noise. Some screams and clapping was taking out though it was
with the music too. A hand drum was mistaken as crowd noise.  Not sure this works
well enough to be effective without doing editing to put back in music or take out scream/whistles.

====================================================================

unmix chorus:

60 min SBD 24/96 wav set for unmix chorus to process 53m.

Took a previous vocal stem with lead singer and harmony by the band and
ran it with unmix chorus. Output layers lead and backup were created.
Lead vocal came out pretty well though sometimes it would get confused
with backup and lead and it would bleed through. Not sure this works well
enough to be effective without doing more editing to add whatever is missing
to another layer or the layer you are working in.
====================================================================

unmix song

9 min song 16 44 wav fareed - 2 acoustic guitar -  5 min to process

Only did 2 layers guitar and other. The guitar layer did not separate the 2
acoustic guitars and the other layer contained a fair amount of guitars when
they both guitars were playing loud.

=========

unmix song

19 min song 16 44 wav flex band - 3 piece jazz combo - piano, bass and drums stems - took 20 minute to process.

Outputted 4 layers - other, piano, bass and drums. Bass layer had almost nothing
in it music wise. Drums were pretty well represented though high end seemed to be
cut off. Acoustic piano was well done though would bleed into the "others layer'.
Other layer contained music bleed from piano and drums.

===============================================================================
spectralayers 11 commands

open wav file

module > unmix song

file > Export > Layers (creates .wav files for each layer open for the unmix options then import layers into DAW)
Title: Re: A.I. audio stem software
Post by: mccordo on November 20, 2024, 11:05:25 AM
Thanks for posting this. I've been considering a software upgrade and this was just the type of info I needed to help make my decision. Looks like Izotope RX is the answer for me.
Title: Re: A.I. audio stem software
Post by: checht on December 04, 2024, 03:45:56 PM
OK, moved my post from another thread here:

I separate out vocals on most every recording. For SBD vocal feeds, it's a great way to debleed.
For aud 2 track recordings, I separate out vocals and mix them back in in parallel to add presences.
Really cures that 'distant' sound that plagues aud recordings. I find it especially helpful on recordings made with Neumann kmi84s from back in the day.

I've been using RX, currently on v9. Today I tried ultimate vocal remover and I'm shocked by how much better it works. Output is much cleaner intem of not clcutting off the beginning of phrases, and not including sax or guitar that would make it through rx. Also, when listening to it solo, the uvr track is drastically more musicall and natural sounding. To me, RX has always sounded kinda alien on its own.

uvr runs much faster than rx on my mac mini m3. And it's free.

What do others use stem separation for, what softwar do you use, and what are your thoughts?
Anyone else compare outputs?

Just uploaded samples to dropbox. There's the original sbd vocals feed, full of stage bleed. Then 2 configs of uvr, then rx music rebalance.
Thoughts?
:
https://www.dropbox.com/scl/fo/izqcnyfgo1kmjc4utaru7/ADXknEXvzjmA0vVEmr4OjLc?rlkey=z1j830ojjrxkecnevykzb2c5i&dl=0
Title: Re: A.I. audio stem software
Post by: nulldogmas on December 04, 2024, 06:02:55 PM
What do others use stem separation for, what softwar do you use, and what are your thoughts?

I use stem separation mostly to rebalance different instruments/vocals, or very occasionally to apply EQ to one but not the others. (Say, if a kick drum is too loud but I don't want to reduce the bass guitar in the same frequency range.) I don't use it on most recordings, but maybe 20-30% of them?

I've tried both RX and UVR and agree that UVR does a better job isolating vocals, though they're both very good for most uses. (I use the MDX23C-InstVoc HQ setting, at Rob G's suggestion.) I haven't tried UVR for multi-stem separation yet — still waiting to hear reports on which settings people find work best.

In either case, I usually export the stems and then remix them in Audacity, where it's easier to play with the sliders on the fly.
Title: Re: A.I. audio stem software
Post by: jefflester on December 04, 2024, 06:28:50 PM
I did a show this weekend with my band, individual instruments > F8, only output available from the board was an FX send and I got a lot more piano and acoustic guitars than vocals so I want to try UVR to pull out just the vocals. What should I use for "Overlap"? DLing the "MDX23C-InstVoc HQ" just now.
Title: Re: A.I. audio stem software
Post by: robgronotte on December 05, 2024, 04:08:48 AM
I did a show this weekend with my band, individual instruments > F8, only output available from the board was an FX send and I got a lot more piano and acoustic guitars than vocals so I want to try UVR to pull out just the vocals. What should I use for "Overlap"? DLing the "MDX23C-InstVoc HQ" just now.

I use UVR5 often, mostly to remove crowd noise from instrumental portions of songs.  I clip out the portion I want to clean up, run it through UVR5, and then patch back in the clean option.  As noted above, the best one I have found is "MDX23C-InstVoc HQ", which has to be downloaded separately (also free and very easy).
I don't know what the "Segment Size" and "Overlap" options refer to, so I have just left them at the default, which was 256 and 8.  If anyone knows more about options on UVR5 I would love to understand it better.

Actually I almost always run in "Ensemble Mode" [with Max Spec/Min Spec setting] which runs the file through several different filters at the same time.  In addition to the one mentioned above, I use "VR Arc1_HP-UVR" and "Demucs v4|htdemucs" (both included in main download).  It barely takes any longer than using the MDX23C alone, as that one is very slow compared with most of the algorithms.   I get 4 outputs - one for each of the 3 processes, plus a combined version.  If the MDX23C version doesn't sound perfect to me, I will listen to the others and possibly use one of those instead.

If anyone else wants to use UVR5 for similar results, feel free to ask any questions and I could give some more detailed info about what I do and the results I get.
Title: Re: A.I. audio stem software
Post by: phil_er_up on December 05, 2024, 06:07:41 PM
Was not sure if this thread would gain any traction so have not posted any additional info. No expert in this just posting obervations/thoughts. Will not say anything about the below files till other listen and give their opinion.
(Links good for a week)

==================================================

Samples:

1) 2 song - 24/96 sbd wav files and created 4 stems - vocal, bass, drum and "other instruments" in RX, UVR and Steinberg Spectral Layers (SL).

2) Linked original 24/96 wav file.

3) Created 6 stems of the same original file in SL (6 stems Vocal, bass, drum, other, piano and guitar) for others to compare how it splits the piano and guitar into separate stems.

Did no processing except take the 16 bit UVR and created 24/96 files so could compare them in my daw. Created flacs from the wavs to conserve file space.   

==================================================

 2 songs - 24/96 sbd wav files and created 4 stems - vocal, bass, drum and "other instruments" in RX, UVR and Steinberg Spectral Layers (SL)

Original 24/96 wav file -
https://www.transfernow.net/dl/20241205Of8Qbegm/A5Cyax6n

RX - 4 stems - vocal, bass, drum and "other instruments" -3 min 9 seconds to create 4 stems
https://we.tl/t-J0a6citpHC

UVR- 4 stems - vocal, bass, drum and "other instruments - 2 min 52 seconds to create 4 stems (Used "htdemucs" model to process the 4 stems)
https://we.tl/t-iZ7LAC1zi1

SL - 4 stems -  vocal, bass, drum and "other instruments - 8 minutes to create 4 stems
https://we.tl/t-zP1ihW2Llb

==================================================

SL - 6 stems Vocal, bass, drum, other, piano and guitar. - 8 minutes to create 6 stems
https://www.transfernow.net/dl/20241205Bb9h9fvL/gxaQu3kr

===========================================

Any comments?
Title: Re: A.I. audio stem software
Post by: robgronotte on December 05, 2024, 06:58:18 PM
Phil, have you tried the Spectralayers function of separation of different vocals?

I tried it once but didn't understand how I was supposed to "train" it to learn the main vocalist.
Was hoping it could cut out audience chat over the singing.
Title: Re: A.I. audio stem software
Post by: phil_er_up on December 06, 2024, 06:35:40 AM
Phil, have you tried the Spectralayers function of separation of different vocals?

I tried it once but didn't understand how I was supposed to "train" it to learn the main vocalist.
Was hoping it could cut out audience chat over the singing.
If I understand what you want to do correctly...then suggest doing the following procedure in SL.

=====================================================================

Create vocal stem and open it in SL.

Select "unmix Multiple Voices" in the modules.

Use cursor and Highlight a 20 second piece or so for main vocalist in the vocal stem
Then in the  "unmix Multiple Voices" window Click "Register Voice" - In SL it creates "voice 1"  in the "unmix Multiple Voices" window.

Now you can select the second vocalist and do the same as above and the then this will create a "Voice 2" for the second vocalist.

Now click "apply" in the  "unmix Multiple Voices" window and SL creates layers for "Voice 1", "Voice 2", "Non_voice" and  "Non-Un-mixed".

File > Export > Layers

=======================================================================
Hope you can understand what I wrote.

=======================================================================

SpectraLayers 11 vs Ultimate Vocal Remover:
https://www.youtube.com/watch?v=u3Ntpdt-pCU
Title: Re: A.I. audio stem software
Post by: robgronotte on December 06, 2024, 12:58:11 PM
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Title: Re: A.I. audio stem software
Post by: phil_er_up on December 06, 2024, 01:06:18 PM
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.
Title: Re: A.I. audio stem software
Post by: robgronotte on December 06, 2024, 02:26:54 PM
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.

How does that work?
Title: Re: A.I. audio stem software
Post by: phil_er_up on December 07, 2024, 07:18:09 AM
So you need to have a portion with each vocalist singing alone in order to use the function?
That seems not very useful, as you would rarely have that available for the backing vocals.
Yes you do.

There is "unmix chorus" too. That splits main singer from chorus.

How does that work?
From what I have seen so far:

unmix Multiple Voices - is to separate 2 or more vocalist to have one vocal layer for each singer.

Unmix chorus -  Output layers lead and backup are created. Seems this would be good choice if had one main singer and then harmony with rest of band.

===================================================

unmix chorus:

60 min SBD 24/96 wav set for unmix chorus to process 53m.

Took a previous vocal stem with lead singer and harmony by the band and
ran it with unmix chorus. Output layers lead and backup were created.

Lead vocal came out pretty well though sometimes it would get confused
with backup and lead and it would bleed through.

=======================================================

The wetransfer links above end tomorrow. Hope someone DL'ed them and has some comments. Gives a good idea of what this software can do.

Title: Re: A.I. audio stem software
Post by: ballerusk on December 18, 2024, 04:06:15 PM
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.
Title: Re: A.I. audio stem software
Post by: AbbyTaper on December 18, 2024, 04:23:10 PM
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.

I tried it once, and would describe the results as "mixed".
Title: Re: A.I. audio stem software
Post by: robgronotte on December 18, 2024, 10:55:21 PM
Hope they have a new year sale or something on Spectralayers. That unmix crowd feature seems golden for audience mumbling chatter.

I tried it once, and would describe the results as "mixed".

I have found the whole program very difficult to use. Very disappointing, because the features seem great.
Title: Re: A.I. audio stem software
Post by: lavaux on December 23, 2024, 09:10:31 PM
Based on quick research, seems like SpectralLayers was built off of Spleeter, which is the old method of extracting elements. They now use Demucs, which you get for free with UVR (but UVR doesn't have the vocal learning feature that Spectralayers has.

I think MVSEP and X-Minus are the most cutting edge, with UVR right behind them. I use UVR to remove audience, then load it up into Izotope RX and spectrally remove all the talking/coughing/non-applause.

Here's a really great research document that showcases all the current updates and movements in the AI audio separation field:
https://docs.google.com/document/d/17fjNvJzj8ZGSer7c7OFe_CNfUKbAxEh_OBv94ZdRG5c/edit?usp=sharing (https://docs.google.com/document/d/17fjNvJzj8ZGSer7c7OFe_CNfUKbAxEh_OBv94ZdRG5c/edit?usp=sharing)
Title: Re: A.I. audio stem software
Post by: if_then_else on December 24, 2024, 03:40:02 AM
There are a few other options that haven't been mentioned in this thread yet.

Title: Re: A.I. audio stem software
Post by: phil_er_up on December 24, 2024, 06:59:03 AM
Have used RX, UVR and SL using many different files and genres. Used the Demus in UVR and found that SL and UVR sounded the best in stem separation and almost similar. RX sometimes has this metallic sound to it. Though found mixed results depending on the genre I used with each software. With RX and UVR they just split stems and that is it. No editing features. SL has these editing features like RX with the addition of the eraser tool, batch processing, select similar and ability to add layers or extract from layers and add to another layer or create a new layer. This is a big step IMO.

Have a fast PC - time to process 70 minute set in following software with 4 stem extraction:
RX - 8 minutes
UVR - 8 minutes
SL 60-70 minutes - you can get around the slow processing time by using the batch processing tool and run it over night.

Tried this over the weekend took a audience recording with crowd noise in the back ground and ran "unmix crowd noise" in SL. Was surprised  how well it took out the chatter from the floor noise. What was left after extracting the crowd noise almost sounded like a SBD though some vocals were taken out with the crowd noise. Then selected the vocals manually in a short section in the crowd noise layer and  ran "select similar". It ran for hours to try and find all of them. Did an ok job. If you extracted just the vocal then ran "unmix crowd noise" it might make a nice sounding audio file. SL is like photoshop and can do many more features then just splitting the files. Though as robgronotte suggests its not as easy program to use and would take time to figure out how to really use it.

====================================================

Thanks to  lavaux and  if_then_else for their input and comments. It is appreciated.

Will look into the software you guys posted about. Thank you!
Title: Re: A.I. audio stem software
Post by: robgronotte on January 10, 2025, 09:52:25 PM
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.
Title: Re: A.I. audio stem software
Post by: phil_er_up on January 12, 2025, 11:07:08 AM
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.
Good job figuring out how to use the tools. Cool that you used SL then UVR5 then SL.

That is my thought pattern too. Will have to use the tools together to get what you want. RX and UVR5 are good at separating a file into stems though that is all they do. SL does so much more though you have to use their tools then manually edit the output to separate what you want in the file and (Noise, chatter, instruments) that you want to take out.

Tried "unmix crowd noise" and thought it did a good job. It completely took out chatter between songs. Though in some songs it took out some of the vocals that the program thought was crowd noise. Wish there was a way to only select different sections of the file then run the tool.



 
Title: Re: A.I. audio stem software
Post by: AbbyTaper on February 01, 2025, 04:37:34 PM
Has anyone tried Moises?  Not free, but under $3 per month on an annual contract.
Title: Re: A.I. audio stem software
Post by: robgronotte on February 01, 2025, 07:26:44 PM
Well, I still haven't figured out a lot of what can be done with SpectraLayers 11, but I just tried using the "unmix crowd noise" for the first time with excellent results.

I took a 14 second part of a recording where there was a lot of crowd chatter under the main vocals and ran it through UVR5.  Took the vocal file from there and ran the unmix crowd noise in SL11, and it gave me a pretty much perfect separation between the singer and the background chatter.  It also was very quick doing that, although again it was only 14 seconds.   Then I exported the "Foreground" layer, and mixed it back with the instrumental audio file from UVR5.

I'm hoping I can use this method in the future instead of agonizingly trying to remove chatter from vocal sections using the spectral repair feature of iZotope RX.
Good job figuring out how to use the tools. Cool that you used SL then UVR5 then SL.

That is my thought pattern too. Will have to use the tools together to get what you want. RX and UVR5 are good at separating a file into stems though that is all they do. SL does so much more though you have to use their tools then manually edit the output to separate what you want in the file and (Noise, chatter, instruments) that you want to take out.

Tried "unmix crowd noise" and thought it did a good job. It completely took out chatter between songs. Though in some songs it took out some of the vocals that the program thought was crowd noise. Wish there was a way to only select different sections of the file then run the tool.

Seems like there should be a way to only select part of a file but I haven't found it. I cut the recording into pieces with another program and then run through just the part I want to use it on. Same thing for UVR5.
Title: Re: A.I. audio stem software
Post by: bmubart on February 04, 2025, 06:15:12 AM
This is my workflow for noisy crowds (for what it is worth):
- Spectralayers: Unmix Song in Voice + Other (= 2 layers)
- Unmix the Voice further with Unmix Crowd.  You become 2 new layers Foreground + Crowd

You can now re-build the whole track in your DAW (Reaper...) using the layers:
1) Other
2) Foreground
3) Crowd

Listen to the result. 
You can now edit the Crowd-layer on the fly in eg Izotope RX or a wave editor to eliminate crowd noise.
Render
--> same recording with almost no crowd

This is very time consuming but the result is very pleasing (imho).
This doesn't work well when there are singers in the crowd which voice blend with the lead-singer

All unmixing in Spectralayers can be done in batch




Title: Re: A.I. audio stem software
Post by: phil_er_up on February 06, 2025, 06:36:38 AM
Good work  bmubart. Thanks for sharing.

As TS members figure out how to use these software's  then maybe they could share their processes so others can learn from them.

SL is an advanced software and getting it to do what you want will take practice and time.
Title: Re: A.I. audio stem software
Post by: bmubart on February 06, 2025, 09:28:03 AM
Good work  bmubart. Thanks for sharing.

As TS members figure out how to use these software's  then maybe they could share their processes so others can learn from them.

SL is an advanced software and getting it to do what you want will take practice and time.

Thanks!

I'm a new user to SL too but it is really not that hard at all for the things I need it: making layers/stems.

I do these steps in batch mode (overnight):
- select the tracks I want to unmix
- select Unmix Song and Seperate them in Layers.  You can choose how many layers (drum, bass, guitar, vocals...) but mostly 'voice' and 'others' will do so each track becomes two wave-files
- select the folder where the batch job has to export the layers (= wave files)

Then: go to the folder with the unmixed layers (= wave file per stem)
- in batch (again):
- choose the voice-layers
- choose 'Unmix Crowd' and select the folder to export the voice layers: Crowd + Foreground

After that you will become all the stems with the voice seperated in two layers: crowd + foreground
Take these stems to rebuild the full track in your DAW:

- others
- foreground
- crowd

(so don't use the original voice-layer)

Play this 3 together in your DAW (= full track WITH crowd-noise)
Eliminate the crowd noise by playing with volume envelopes or by reducing it / deleting it in your wave editor (I use Izotope RX)
Listen again = full track with less crowd noise

Hope this helps?


Title: Re: A.I. audio stem software
Post by: checht on February 06, 2025, 11:15:32 AM
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?
Title: Re: A.I. audio stem software
Post by: nulldogmas on February 06, 2025, 03:54:17 PM
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?

I'm on a Mac, but haven't tried ensemble yet. What stems are you choosing?
Title: Re: A.I. audio stem software
Post by: phil_er_up on February 08, 2025, 12:22:27 PM
Anyone else working on a Mac?

I'm having issues with UVR using up all memory if I choose ensemble, then crashing the machine.

Anyone?
Hi Chris,

On old windows PC without a graphics card it would take hours to run UVR with 4 stems and a 2 GB wav file and sometimes would hang my pc.
On new fast win 11 pc with graphics card and 8 gb memory takes only 8 minutes to process 4 stems with 2 GB wav file.

Believe you told me it was only 8 minutes to process one stem on your mini-mac thought that was really fast though was surprised due to you did not have a graphic card.
Have looked at task manager in win 11 when UVR is running and my graphics card is running at 100%. Though it does eat memory too. If creating 4 stems with a 2 GB file you need 4-2gb files output that is 8 Gb in file output from UVR.

So assumed you needed a graphic card for UVR to run fast and not crash your machine.
Title: Re: A.I. audio stem software
Post by: checht on February 09, 2025, 01:15:42 PM
Hiya Pat,

Except for the Mac Pro, Macs don't use graphics. cards these days.

I'm on the road w Los Lobos, so using Macboook Air and just going with no ensemble, just MXR Main. 1.25 hour show takes around 45 min to process. In this application, I'm recording omt4 with mk41s x/y PAS in the center, and mk22s 4' spread. I'm not too picky about extraction in this case, as I'm pulling the vocals from the 41s to mix back in in parallel with the 2 stereo tracks, a variety of vocal sweetening.

Here's a track from Thursday night:
https://archive.org/details/LosLobos2025-01-06/ll20250206.OMT4.2448-02.flac

This is my current state of the art, combining AI extraction with omt4 recording. Fun times.

Not very good picture attached showing omt4 boom at the triple door in Seattle.
Title: Re: A.I. audio stem software
Post by: robgronotte on July 12, 2025, 10:43:58 PM
SpectraLayers 12 was recently released.  It's supposed to have better stem separation and definitely has more individual stems it can create.  A small test with the Unmix Crowd Noise module showed no improvement for me, sadly, as that is all I've really used SL11 for.

Has anyone managed to successfully use the Unmix Multiple Voices module?  I've never been able to figure it out with any success.
Title: Re: A.I. audio stem software
Post by: rastasean on July 19, 2025, 03:23:35 PM
Here's another project you folks may like: https://github.com/CarlGao4/Demucs-Gui

You're able to separate out all kinds of stems and even make your own mix.
Title: Re: A.I. audio stem software
Post by: phil_er_up on July 23, 2025, 01:52:38 PM
Has anyone managed to successfully use the Unmix Multiple Voices module?  I've never been able to figure it out with any success.
Create vocal stem and open it in SL.

Select "unmix Multiple Voices" in the modules.

Use cursor and Highlight a 20 second piece or so for main vocalist in the vocal stem
Then in the  "unmix Multiple Voices" window Click "Register Voice" - In SL it creates "voice 1"  in the "unmix Multiple Voices" window.

Now you can select the second vocalist and do the same as above and the then this will create a "Voice 2" for the second vocalist.

Now click "apply" in the  "unmix Multiple Voices" window and SL creates layers for "Voice 1", "Voice 2", "Non_voice" and  "Non-Un-mixed".

File > Export > Layers
Title: Re: A.I. audio stem software
Post by: hoserama on July 29, 2025, 01:06:12 PM
Here's another project you folks may like: https://github.com/CarlGao4/Demucs-Gui

You're able to separate out all kinds of stems and even make your own mix.

Ultimate Vocal Remover can do everything that does, and more.
Title: Re: A.I. audio stem software
Post by: hoserama on July 29, 2025, 01:09:33 PM
As an aside, Apple's new models included in the latest version of Logic Pro are easily the best I've played with. It does a really good bass/drums/vocals/guitar/piano/other. In particular, the guitar and piano stems are far better than any others I've used.
Title: Re: A.I. audio stem software
Post by: al w. on July 29, 2025, 04:24:54 PM
As an aside, Apple's new models included in the latest version of Logic Pro are easily the best I've played with. It does a really good bass/drums/vocals/guitar/piano/other. In particular, the guitar and piano stems are far better than any others I've used.

+1, the new version is superb!
Title: Re: A.I. audio stem software
Post by: robgronotte on July 30, 2025, 01:17:58 AM
Here's another project you folks may like: https://github.com/CarlGao4/Demucs-Gui

You're able to separate out all kinds of stems and even make your own mix.

Ultimate Vocal Remover can do everything that does, and more.

What model / settings do you usually use in UVR5?

The one I've generally gotten the best results with is MDX-NET MDX23C-InstVoc HQ, but it's about two years old now.  I can't find any more recent recommendations anywhere.
Title: Re: A.I. audio stem software
Post by: phil_er_up on July 30, 2025, 06:48:45 AM
As an aside, Apple's new models included in the latest version of Logic Pro are easily the best I've played with. It does a really good bass/drums/vocals/guitar/piano/other. In particular, the guitar and piano stems are far better than any others I've used.
What Apple new models are you referring too?
Do you have a link?
Title: Re: A.I. audio stem software
Post by: ballerusk on July 30, 2025, 07:01:45 AM
https://support.apple.com/no-no/guide/logicpro/lgcp61bae908/mac

Stem Splitter is available on Macs with an M1 or later Apple silicon processor.
Title: Re: A.I. audio stem software
Post by: hoserama on July 30, 2025, 04:18:49 PM
Here's another project you folks may like: https://github.com/CarlGao4/Demucs-Gui

You're able to separate out all kinds of stems and even make your own mix.

Ultimate Vocal Remover can do everything that does, and more.

What model / settings do you usually use in UVR5?

The one I've generally gotten the best results with is MDX-NET MDX23C-InstVoc HQ, but it's about two years old now.  I can't find any more recent recommendations anywhere.

I was typically using the htdemucs_ft models, and would sometimes run it along side the mbf_kim_vocals (I forget exactly the name). Now I'm using the Logic models as my mains.
Title: Re: A.I. audio stem software
Post by: hoserama on July 30, 2025, 04:20:08 PM
As an aside, Apple's new models included in the latest version of Logic Pro are easily the best I've played with. It does a really good bass/drums/vocals/guitar/piano/other. In particular, the guitar and piano stems are far better than any others I've used.
What Apple new models are you referring too?
Do you have a link?

Right now its only thru the Apple Logic Pro program. Friend of mine was able to extract it and testing out a GUI for that in windows, but still very much in the beta form.
Title: Re: A.I. audio stem software
Post by: guitard on July 31, 2025, 06:42:28 AM
This guy was on Eddie Trunk recently.  Pretty cool stuff he's doing with stems.

https://www.youtube.com/@franKENstein-Creations
Title: Re: A.I. audio stem software
Post by: ballerusk on August 01, 2025, 03:52:50 PM
Has anybody seen any tutorials (videos or written) on mixing/mastering audience recordings? Everything I have found has been on soundboard or studio recordings where they have each instrument/element separated.

I played around with stems from UVR but it seems ..."wrong" working on something I feel is not 100% clean (some bleeding between stems, stems not getting all vocals, drums, etc) and that it would be better to mix/master on the recording as a "whole".
Title: Re: A.I. audio stem software
Post by: robgronotte on August 02, 2025, 01:28:00 PM
I don't know of anything like that, but I have posted fair amount about my process here in this thread and maybe another in this sub-forum. I'm happy to answer any questions as well.
Title: Re: A.I. audio stem software
Post by: ballerusk on September 03, 2025, 01:19:08 PM
Not exactly stem separation, but the new Ozone 12 (advanced) has a new feature called "stem EQ": https://www.izotope.com/en/products/ozone/features/new-stem-eq

"EQing the vocals, bass, drums, and instruments in parallel in your stereo bounce."

No idea if the stem separation is better than UVR, but maybe easier to work with when you only need to EQ and not remove whistles, talkers and other fixes usually done in RX.
Title: Re: A.I. audio stem software
Post by: nulldogmas on September 03, 2025, 01:49:07 PM
Not exactly stem separation, but the new Ozone 12 (advanced) has a new feature called "stem EQ": https://www.izotope.com/en/products/ozone/features/new-stem-eq

"EQing the vocals, bass, drums, and instruments in parallel in your stereo bounce."

No idea if the stem separation is better than UVR, but maybe easier to work with when you only need to EQ and not remove whistles, talkers and other fixes usually done in RX.

So you can adjust EQ and gain on the stems but not export them individually? That'll work, but it's a weird limitation.
Title: Re: A.I. audio stem software
Post by: ballerusk on September 03, 2025, 01:59:07 PM
Well, it saves you separating the stems and then EQ each track. Now you can import the stereo track and EQ the stems in one track. I guess exporting individually is more a RX feature.
Title: Re: A.I. audio stem software
Post by: nulldogmas on September 03, 2025, 06:46:04 PM
Well, it saves you separating the stems and then EQ each track. Now you can import the stereo track and EQ the stems in one track. I guess exporting individually is more a RX feature.

Right, you can do either of those currently in RX. So this looks to be a crippleware version of the RX stem feature, which, okay.
Title: Re: A.I. audio stem software
Post by: lpmaskman on September 06, 2025, 05:50:34 AM
New Ableton Live will feauture stem separation too: https://www.ableton.com/en/blog/live-12-3-is-coming/