Sometimes men (or women) just want to have a quick jerk without "the hassle" (foreplay, pleasing the partner as well, etc.) of sex. That's perfectly normal and shouldn't be shamed. If he however turns you down alot and you feel rejected, you should have a serious conversation with him about ways to spike your sex life up.
Men aren't always in the mood either and maybe he just needs a different atmosphere, like a romantic evening or some toys? Lots of things you could try together :)