Topic: AI stems separation: piano vs. strings or orchestra

[WiFiJeff]

I have searched the threads here, googled online, and used Rx10's music rebalance, but none of these resources seem to address the issue I have, namely separating and rebalancing piano sound from an otherwise good stereo recording.  Most discussion seems to be concerned with eliminating audience noise or dealing with vocals or drum tracks.  My issues involve chamber music, where the piano is too recessed, or piano/orchestral concertos, where it is too forward.  Since my ears can differentiate piano sound vs. strings I suppose it is possible for AI to differentiate too, but of course the frequencies overlap and the longer attack and decay of each note will be important.   Does anyone know of a good tool for this kind of work??

[robgronotte]

SpectraLayers claims to do this well.  I've never had the need, but it is pretty good with other things I use it for.  If you want to send me a song to test, I can run it through and give you the results.

[WiFiJeff]

SpectraLayers claims to do this well.  I've never had the need, but it is pretty good with other things I use it for.  If you want to send me a song to test, I can run it through and give you the results.

pm sent

[Billy Mumphrey]

I had a need for separating piano (and horns) in quite a few recordings lately, although not with strings.

I purchased Spectralayers 12 through Black Friday sales about a month back.

I've also been using UVR5 (freeware!!) and some of their latest AI models have the 6 stem separation that we seek (drums, bass, vocal, piano, guitar, other).

I have been sooo busy lately and have not had time to properly compare, but I will say both performed somewhat similarly in my limited tests. As many who use stem software have found out, the higher quality / clarity of your initial recording determines how good your individual stems will be. But since UVR5 is free, it's a not a terrible idea to start there. I got Spectralayers for $209, and I think its normally around 300. Perhaps you can try UVR5 until a new sale happens, I've been working 2 jobs lately so I had a little extra cash and I've read about Spectralayers crowd noise removal so I decided to make the purchase.

As others have reported in the AI stem thread in this sub forum, UVR5 is a bit weird to use, but definitely not impossible. It's confusing and somewhat overwhelming navigating through the different models and how to apply them. The software also seems to run through your GPU (or it at least has settings for it) so a higher end graphics card is at least recommended (but not completely necessary).

Keep in mind a couple people have posted here that Logic Pro's latest AI stem models are quite good, and I think said they appeared to be an improvement on all the previous software options (please confirm or deny if you read this). Logic requires an Apple computer, and I'm seriously considering picking up a Macbook or Mac mini to try this out.

If you're computer saavy then definitely download UVR5 asap, and if you're not then try it anyway haha.

[nulldogmas]

I have been sooo busy lately and have not had time to properly compare, but I will say both performed somewhat similarly in my limited tests. As many who use stem software have found out, the higher quality / clarity of your initial recording determines how good your individual stems will be. But since UVR5 is free, it's a not a terrible idea to start there.

Do you have a recommendation of which UVR5 model worked best for stemming multiple instruments?

[WiFiJeff]

SpectraLayers claims to do this well.  I've never had the need, but it is pretty good with other things I use it for.  If you want to send me a song to test, I can run it through and give you the results.

Thanks very much for this.  I took the "extreme" files and played with them a bit, not done yet.  The "piano" file got most of the piano sound okay, I'm not sure how the violin, viola and cello made it into the other files (guitar??, other?? Sax and Brass??!!)!  I converted the piano flac file back to 24/96 wav files and mixed it with my original first without any level change and then up 6 dB, that did bring the piano out nicely.  I will revisit carefully tomorrow because I think there is some overall change to the sound (other than the piano popping out) which does not feel as nice.  Maybe I can't use my original directly but have to blend the three other tracks you sent back with the piano but at lower levels??  I'm new to this!

Thanks very much for helping me assess the current state-of-the-art AI, clearly it is tending in the right direction.

Jeff

[phil_er_up]

Do you have a recommendation of which UVR5 model worked best for stemming multiple instruments?

If you doing only 4 stems try the "Demus" then select "Htdemus" for 4 stems  Vocal, bass, drums and other. This model sounds quite good.

If you want more then 4 stems in UVR use "Demus" then select "Htdemus_6" models and it will give you 6 stems - Vocal, bass, guitar, drums, piano and other. Though this option does not separate it as well as SpectraLayers IMO. Found the Htdemus_6 does not sound as good as the Htdemus model.

Found if only want vocal use the "MDX-net" and select "vocals" check box has the best output of vocals only between UVR, SpectraLayers and RX .

IMO you need a graphics card with lots of memory or UVR will run really slow. Can process a 2 GB 24/96 wav file in less then 5 minutes with UVR5 with my graphics card and 8 GB of memory. Files come out in 16 bit only so have to adjust that file for my DAW if original wav file is 24 bit.

-------------------------------------------------------------------------------------
SpectraLayers works better at stem separation then UVR5 except for vocal only I have found. Did not think RX does a better job then UVR5 or SpectraLayers. RX has a metallic sound to it IMO.

None of these are perfect. There is some bleed through with other instruments. Have never tried piano stem before so not really can comment on how those stems sound.

----------------------------------------------------------------------------------
When putting these stems back together sometimes I use the original wav file plus the stems in my DAW and adjust each one to my liking. Good thing is you can run different plugins on each stem to achieve what you are going after IMO. Then you can add/subtract  gain on each stem too. Adds more flexibility overall with stems.

----------------------------------------------------------------------------------
Hope others can understand what I wrote. There are so many options best to try different software and models and see what your experience is with them.

---------------------------------------------------------------------------------
To WiFiJeff if you want me to run one for you and send you the stems PM me. Happy Stemming!

[nulldogmas]

Do you have a recommendation of which UVR5 model worked best for stemming multiple instruments?

If you doing only 4 stems try the "Demus" then select "Htdemus" for 4 stems  Vocal, bass, drums and other. This model sounds quite good.

If you want more then 4 stems in UVR use "Demus" then select "Htdemus_6" models and it will give you 6 stems - Vocal, bass, guitar, drums, piano and other. Though this option does not separate it as well as SpectraLayers IMO. Found the Htdemus_6 does not sound as good as the Htdemus model.

Found if only want vocal use the "MDX-net" and select "vocals" check box has the best output of vocals only between UVR, SpectraLayers and RX .

Thanks, good to know! I've used MDX-net previously, will try the Demucs options.

[hoserama]

Keep in mind a couple people have posted here that Logic Pro's latest AI stem models are quite good, and I think said they appeared to be an improvement on all the previous software options (please confirm or deny if you read this). Logic requires an Apple computer, and I'm seriously considering picking up a Macbook or Mac mini to try this out.

Can confirm on this point. Lots of experience using various models (particularly demucs) in UVR, Spectralayers, and RX. The new logic models smoke it. The piano and guitar models are pretty damn nifty, compared to the 6-stem demucs (which never claimed to have the piano/guitar models to be anything more than a beta).

[Billy Mumphrey]

Do you have a recommendation of which UVR5 model worked best for stemming multiple instruments?

Similiar to phil_er_up, I use the "Demucs" processing method, and the v4 "htdemus_6s" model. A cool (albeit confusing) part of UVR5 is that there are newly AI updated models coming out and you can download them through the software. There are like 5 different processing methods, and I haven't the time (or patience) to sort through them though.

If you doing only 4 stems try the "Demus" then select "Htdemus" for 4 stems  Vocal, bass, drums and other. This model sounds quite good.

If you want more then 4 stems in UVR use "Demus" then select "Htdemus_6" models and it will give you 6 stems - Vocal, bass, guitar, drums, piano and other. Though this option does not separate it as well as SpectraLayers IMO. Found the Htdemus_6 does not sound as good as the Htdemus model.

Found if only want vocal use the "MDX-net" and select "vocals" check box has the best output of vocals only between UVR, SpectraLayers and RX .

IMO you need a graphics card with lots of memory or UVR will run really slow. Can process a 2 GB 24/96 wav file in less then 5 minutes with UVR5 with my graphics card and 8 GB of memory. Files come out in 16 bit only so have to adjust that file for my DAW if original wav file is 24 bit.

-------------------------------------------------------------------------------------
SpectraLayers works better at stem separation then UVR5 except for vocal only I have found. Did not think RX does a better job then UVR5 or SpectraLayers. RX has a metallic sound to it IMO.

None of these are perfect. There is some bleed through with other instruments. Have never tried piano stem before so not really can comment on how those stems sound.

----------------------------------------------------------------------------------
When putting these stems back together sometimes I use the original wav file plus the stems in my DAW and adjust each one to my liking. Good thing is you can run different plugins on each stem to achieve what you are going after IMO. Then you can add/subtract  gain on each stem too. Adds more flexibility overall with stems.

----------------------------------------------------------------------------------
Hope others can understand what I wrote. There are so many options best to try different software and models and see what your experience is with them.

---------------------------------------------------------------------------------

Lots of good info here, thanks for this and all the helpful advice in the other stem thread. There are so many options!

Couple points:

-UVR5 apparently requires a NVIDIA gpu if you want to select the "GPU conversion" box. I have a decent graphics card but it's not NVIDIA and I can't select that box, so it uses the CPU (what a crazy concept haha) but it does take a while.
-thank you for mentioning the 16-bit output of UVR5. Like most, I record and edit in 24-bit (or 32fp) so it's annoying that you can't output anything other than 16-bit. Doesn't really affect your final product too much, I've made some great stem-assisted recordings using UVR5 stems. I didn't even convert them, I just dropped them into Reaper multi and starting mixing etc.
-piano stems are tricky, especially when an electronic keyboard synth is involved. Keyboard ends up coming through the "other" stem it seems more often than not.
-When stemming an audience recording, I definitely use the original wav file, then add the stems in a multitrack setting and starting mixing to taste. Aud stems never sound as good as a sbd feed, and I really only use this method when the balance of instruments is off in the original aud recording. I tape at this local spot all the time where the drums are really far back on the stage, but I have a sweet DFC spot right in front of the PA speakers. Everything coming through the PA sounds nice and clear and upfront, but the drums sound distant so I stem and boost them with some EQ etc and it really balances the recordings I make there. Stemming a sbd feed and mixing it in with a solid audience recording can produce some killer results, especially when you learn some mastering techniques. Have some groups considering official releases with this technique.

Keep in mind a couple people have posted here that Logic Pro's latest AI stem models are quite good, and I think said they appeared to be an improvement on all the previous software options (please confirm or deny if you read this). Logic requires an Apple computer, and I'm seriously considering picking up a Macbook or Mac mini to try this out.

Can confirm on this point. Lots of experience using various models (particularly demucs) in UVR, Spectralayers, and RX. The new logic models smoke it. The piano and guitar models are pretty damn nifty, compared to the 6-stem demucs (which never claimed to have the piano/guitar models to be anything more than a beta).

Ah great, now I have to buy a Macbook. 

[phil_er_up]

Couple points:

-UVR5 apparently requires a NVIDIA gpu if you want to select the "GPU conversion" box. I have a decent graphics card but it's not NVIDIA and I can't select that box, so it uses the CPU (what a crazy concept haha) but it does take a while.

Bill did not know that UVR5 requires a NVIDIA gpu. My graphics card must have a NVIDIS gpu because I can select  the "GPU conversion" box. Thanks for sharing.

-thank you for mentioning the 16-bit output of UVR5. Like most, I record and edit in 24-bit (or 32fp) so it's annoying that you can't output anything other than 16-bit. Doesn't really affect your final product too much, I've made some great stem-assisted recordings using UVR5 stems. I didn't even convert them, I just dropped them into Reaper multi and starting mixing etc.

Use wavelab and it will not let me put 24 and 16 bit files in the same montage though in reaper you can mix 24 and 16 bit files and not have to convert them to 24 bit unless you need/want too.

---------------------------------------------------------------------------------

Do not want to buy a mac just for processing then I need to buy the software too. Many times other software will come out with something else in a short period of time or that is my hope.

Responded to wifijeff in a PM.

---------------------------------------------------------------------------------
hoserama if you have more to share or experience would love to hear about logics stem software.

[Billy Mumphrey]

Yeah that's a good point, there's a solid chance the other software companies will catch up to Logic.

[nulldogmas]

I just downloaded a free trial of Logic, and my first tests of stem splitting are indeed impressive. I'll likely wait to see if RX upgrades its stem function as well, but this is really promising in terms of where the technology is headed.

[hoserama]

Nvidia - the AI stemming models use the Nvidia CUDA toolkit. So you'll need a compatible nvidia card to supercharge the processing...makes a huge difference in terms of timing. 2 hour show for the htdemucs_ft model typically took 8-9 hours when I was just running on CPU...takes 20 minutes with my Nvidia card.

16-bit/24-bit - My UVR has a setting to allow for 24 bit.

Logic - Buddy of mine has a beta program where he extracted the Logic Pro models (and a few others) and we've been running it in windows. Still in development but it can be done outside of the mac environment. I've stopped using demucs ever since getting the LogicP models running. I'll make a post on here when it's ready for primetime.

[WiFiJeff]

First of all, major thanks to robgronotte and phil_er_up for the SL runs and the UVR, the first reaction I have gotten is amazement.  I have currently sent the remixed files to a number of piano and violin friends, the initial impression is always the same as mine: first order a great improvement, no question.  The only issue is potential "ear fatigue" and the attachment I have to what my mics produce naked, as heard through my editing headset.  Are the changes to the overall sound (piano balance aside) neutral or bad (or maybe even also positive, if heard by others without my bias to what I remember from the hall). 

I am resisting the urge to move to Logic Pro (and Mac, with the expected horrendous learning curve of a shift in operating systems) in hopes that Rx or others play catch-up really fast.

Jeff