re: sex with a married man--would you have you or would you never...
Whether it's a straight relationship or a gay one, domesticity has always been the natural enemy of passion and excitement.
Your married man with his wife of five years, probably did it far more often at first until daily life settled on paying bills, deciding what to have for dinner, fighting over money, maybe dealing with kids (you don't mention), appliances that bust for no reason, and maybe leaky drains. He could have had a boyfriend for just as long and now be just as itchy for an untroubled fresh alliance. You are naturally going to be more exciting for him because the two of you don't share those same at-home headaches.
Not even bisexuals who have pledged a commitment are entitled to a lifetime of one from column A and one from column B. The other poster(s) ahead of me who warn you not to fall in love know what they're talking about.
Even if your man left his wife and family for you, it would not likely be smooth sailing.
My late best buddy, at two different times in his life, set up housekeeping with lovers that had just exited hetero marriages with children. In each case, it was obvious that the newly free man sought to replicate the security of 'home' he'd just given up, but soon equally clear that he had not yet really sown his gay 'wild oates,' which ultimately put my buddy's heart behind an 8 ball.
I won't try to talk you out of having your fun, but be careful with your heart.