All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
That expression does not affect the marriage contract whether the husband said it when he was angry or while in normal state. However, it is not permissible for a husband to ask his wife this question, and it is not permissible for her to inform him about it (the truth), even if she really had committed this sin because it is an obligation on a Muslim to conceal himself/herself if he/she had committed one.
Speaking about a sin which someone had committed is considered another sin.
For more benefit, please refer to Fatwa 92038.