All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
You must observe Hijaab in front of that man, and you are not allowed to show anything of your adornment in front of him just because he is old in age, unless he has reached the point of not having any desire in women.
Allah Says (what means): {…and not expose their adornment except to their husbands, their fathers,…..or those male attendants having no physical desire, or children who are not yet aware of the private aspects of women…….} [Quran 24:31]
Ibn Qudaamah said:
"The men who do not have any desire (for women) due to old age or due to impotence, then he takes the same ruling as that of a Mahram in regard to looking (at non-Mahram women), as Allah Says (what means): {… or those male attendants having no physical desire}; meaning who do not have any need for women." [End of quote]
On the other hand, there are some rulings and etiquettes for seeking permission to enter; so it should be clarified to him that he has no right to enter until he asks for permission. Allah Says (what means): {O you who have believed, do not enter houses other than your own houses until you ascertain welcome and greet their inhabitants. That is best for you; perhaps you will be reminded.} [Quran 24:27]
Indeed, the Sunnah clarified that seeking permission (to enter) is three times; if one is given permission to enter then he can; otherwise, he should go back. To simply knock the door and enter directly is not permissible.
Additionally, visitations have their etiquettes; visiting someone so too many times in a manner that disturbs the people of the house is a matter that should not be done.
'Abdullaah ibn 'Amr narrated that the Prophet said: "If you space your visits, you will be liked more." [At-Tabaraani]
Al-Ghazaali said in Ihyaa’ ‘Uloom Ad-Deen:
"What he meant is that one should be moderate in his visitations and when; his visits should not be very rare and they should not be too often. Visiting very rarely leads to deserting someone, and visiting too often leads to boredom." [End of quote]
It is your right as a wife to be in a separate dwelling where you would not experience hardship. It is for this reason that the jurists stated that the wife has the right to be provided with a separate accommodation and that she does not have to live with any of her husband's relatives unless she willing agrees to this. For more benefit, please refer to Fatwa 84608.
You do not have to bear any difficult situation. However, you should talk to your husband calmly, and he should understand your situation and repel harm from you.
Also, for more benefit, please refer to Fataawa 253297, 88654 and 384583.
Allah knows best.