This may be a PA issue more than anything else. I know that when Paco ran sound at the Pour House, this quality would happen every time the band played really loud. When the PA is overdriven, the sound tends to compress, leading to that auditory quality.
I would really not push the SBM. When the lights are on, you still have another 1 db of headroom, but it runs out really quickly. The SBM does clip like everything else when run really hot so I would keep it on the conservative side. When the SBM would clip, it sounds like a bit of static so that, blended with loud vocals, may be what you are hearing.