I'm deeply in love with my boyfriend but he doesn't feel the same?

Okay well I have been with this man for over 5 years. We have known each other as aquaintances since childhood. We started chatting innocently enough over 5 years ago and it got to be that he would call me every day some times several times a day. Before I go any further we are both married to other people. I don't need you all telling me how wrong it is and karma and all that other stuff. I never expected it to go any further. I have been in a very unhappy marriage and always had perfect morals. I never would cheat I would stay in this marriage for my children and make everyone happy and keep my mouth shut. My husband suffers from depression and has been emotionally abusive over the years. Anyway, this other person became one of my best friends and I could not wait for his calls. He would talk to me and we got along great. About 2 years ago it slowly started to become physical and since she went back to work 8 mos ago we have been seeing each other frequently. I have fallen in love with him and thought he felt the same way. I told him I love him. and he said I care so so so so much about you...He has always been very paranoid about his wife finding out. He knows I would leave my husband for him but he tells me since his kids are so young that he can't leave 2 of them under 5. So I have so many emotions...even horrible enough to think that if he leaves me I'll make sure his wife finds out. I don't need a lecture I know it's wrong but I am just so hurt and so confused right now. I don't know what to do. Should I just accept the relationshiop the way it is if I want to be with him or try to leave him. I can't imagine my life without him....Please try to give serious advice and leave the judging out. You can't really judge a person till you walk a mile in their shoes....