The cases I've seen where the mother was unfit usually involved her substance abuse. It definitely happens, but generally mom sort of bows out because she's not fully engaged anyway.

Studies have shown that women who are abused often appear to be mentally ill because who wouldn't be after living in fight or flight mode constantly? And the abuser comes out smelling like roses, because despite what he says about "losing control" he's always FULLY in control.

So yes, it's a huge problem and one that the court system has almost no training on. Luckily my abuser did not even want custody. So he just continually threatened me, but never actually acted on it.