Should I be worried about husband's sudden need to flirt with other women?

Is it just normal husband behavior to need to be with other women. Why is it always women he wants to invite to our place for pizza and a movie. He's left the house at times to go knock on female neighbor's doors to invite them over to watch tv without okaying it with me first. The day when he did that I asked if he minded if I just went in the other room while he hung out with the neighbor. He turned the whole subject around and said that he was only doing it so that I could make new friends. I never expressed any unhappiness with my current friends. I have a girlfriend online and I hope to be close with my sister again. I have my family. I don't see anything wrong with enjoying my solitude. I don't mind him having friends, male or female. In fact, I wish he did go out more just so he could have fun with others while I savor my alone time. When I told him all this, he acted strange. I wish that he would admit that he's the one seeking a female friendship and not me. The fact that he doesn't want to admit that he is needing it makes me wonder why he's ashamed. Should I be worried? I do know that he would not approve at all if I had male friends. In fact, if an elderly male neighbor needs my help, he will sometimes get jealous and accuse me of caring about them more than him. If I ever talked to men the way he talks to women, he'd have a fit.