I get the feeling that this open relationship thing was his idea, and he probably jumped into bed with your ex as soon as you agreed to go along with it. I'm also thinking that it took you a while to find your new boyfriend, and your husband either thought you'd never find someone else, or you wouldn't go as far as sleeping with them. Now that you have, it sounds like he's trying to make you feel guilty for having a boyfriend, even though he agreed to it, because what he really wanted was an excuse to bang your ex without you calling him a cheater, and still have you waiting for him should your ex become unavailable, or interested in a threesome. If this seems familiar, your husband is a dick, and you should consider a separation/divorce. If I'm way off, your husband is still a dick for saying you betrayed him, but you can tell him it was something he agreed to, so he has to live with it just like you.

Everyone in your house should wear earplugs at all times. Problem solved.