I would leave him no matter how much I loved him, the kids come first, and he might hit them one day. Plus, if he hit me once, he may start to hit me constantly!