feeding ?

I've noticed that after my 4 month old son is finished nursing (only the night feedings), he wants keep my nipple in his mouth. He gets upset when I pull it out. I feel like he's trying to use me as a pacifire--which he will not take. So I have to try to comfort him after I take it away and he still tries to turn to my . Did anyone else go through this?