If you aren’t absolutely sure about it then don’t do it.

Jesus Christ!! Just because you write something in your diary DOES NOT MEAN you want to enact in real life, FFS!! Your husband is a sick ****. Do NOT LET HIM make you do anything you don’t want to ACTUALLY want to do!! Fantasy is fantasy!!!

Sometimes something that can be a nice fantasy is also something you don’t actually want to do IRL, and that’s fine. I think your husband is trying to be supportive with this, which hopefully means he will also listen if you tell him no, reasoning optional. I am concerned about him reading your diary though. Does he normally do that? Did you know he wanted to read it and give permission for him to do so? That’s a big invasion of privacy, and you’re allowed to have a space to write down your own thoughts and expect them to stay private unless you decide to share them.

Why was your husband reading your diary? Even though you're married, some things are personal and none of the others business.

He probably has the same fantasy, but didn't know how to bring it up. So seeing it in your diary gave him a green flag. He should have spoke to you about it first though.

Possible outcome #1: You all enjoy it, and decide to carry on every so often (extremely unlikely). Possible outcome #2: You all enjoy it, but all agree it was a one-off and remain good friends (very unlikely). Possible outcome #3: Your husband and his friend become closer, and your husband suddenly starts "working late" (likely). Possible outcome #4: You become closer to your husband's friend, and you suddenly start "working late" (very likely). Possible outcome #5: Shit goes south after the Green-Eyed Monster makes an appearance as the univited fourth member of your group, your husband falls out with his friend, you and he begin to argue all the time, and then one or both of you realize the only option left is divorce (extremely likely).

Never do it with friends. Never do it in an insecure relationship. And never do it when you can't communicate well in effective together. Otherwise: discuss it, talk about your wishes and not every fantasy needs to become reality :)