lacking Music Rebalance:
Mult a second copy of the source in parallel.
Listening only to the copy, gate it so it sounds like drum hits only.
Reverse it's polarity.
Mix it with the original at whatever level reduces drums appropriately. Tweak gate settings to see if any slight variation improves results.
If you ever need more drums, don't reverse polarity.