You misunderstand them. They aren't calling you that because you're a pathetic loser that they can walk all over. It's because you like to stand at the door and greet people and make sure that their shoes are clean.
Trust me, it's not the "pathetic loser" thing. I promise.