Issues

Why is it when asked "are you upset at me?" cant I just say "hell yes Im upset at you" ? Im scared to rock the boat. Im scared that he will tell me to "pack your shit and get the fuck out". Which is very wrong of me cause he is not the one that said that to me. So instead of telling him how I feel I keep it inside which inturn keeps me up most the night with either bad dreams or just not being able to sleep which also means he sleeps like shit too. In the first place if he tells me to leave it isnt like I dont have my own house that I live in most of the week. And if I cant voice my opinion without him telling me to leave then what good is the relationship in the first place. So instead of keeping it inside and letting it all well up and then me going off I need to learn to speak my mind. Telling him 4 days later that it bothered me doesnt work to well.