All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
In principle, a Muslim is without fault and should be given the benefit of the doubt. It is impermissible for a Muslim to accuse his fellow Muslim of wrongdoing without evidence. Allaah, The Exalted, says (what means): {O you who have believed, avoid much [negative] assumption. Indeed, some assumption is sin. And do not spy or backbite each other…} [Quran 49:12]
The impermissibility of unfounded accusations is more emphasized when it comes to the spouses given the sacred bond between them which Allaah has described as a “solemn covenant”. Allaah, The Exalted, says (what means): {And how could you take it while you have gone in unto each other and they have taken from you a solemn covenant?} [Quran 4:21]
In fact, it is an even graver matter that the accusation against a wife involves committing adultery. Please refer to Fatwa 88510.
It is quite strange that the husband does that under the eyes and ears of his children! However, their claim that slander is Kufr (unbelief) is not correct unless their intention is to show its danger and that it is a grave sin.
Anyway, having ill thoughts about people often leads to other evils such as spying. Islam forbids spouses in particular from spying on each other. It has been reported that “The Messenger of Allaah, sallallaahu ‘alayhi wa sallam, forbade a man from entering upon his wife's (unexpectedly) during the night because of doubting her fidelity or spying on her faults.” [Muslim]
You did well by being patient with your husband to protect your family. We advise you to supplicate Allaah for him to guide him and protect him from the evil of his own self and the evil of the devil and his whisperings. Also, try to advise him kindly, revive the fear of Allaah within his heart, and remind him of the danger of thinking ill of others and spying. You may even threaten him to refer the case to someone who is believed able to deter and stop him.
Finally, it is very important that you adhere to the Islamic Hijab and to modest conduct. You are also advised to avoid being in situations that may be misunderstood by others and cause them to have doubts about you. It should be noted that it is impermissible for a Muslim woman to work in a place where men and women intermix freely except out of necessity, as underlined in Fatwa 82399.
Allaah Knows best.