All perfect praise be to Allah, The Lord of the Worlds. I testify that there is none worthy of worship except Allah, and that Muhammad is His slave and Messenger.
If this man married this non-Muslim woman who is neither a Jew nor a Christian, then the marriage is null and void as Allah Says (what means): {And do not marry polytheistic women until they believe.} [Quran 2:221]
Therefore, this man should immediately separate from his wife because this marriage is not valid at all. The children from this marriage are traced back to the father since he copulated with his wife while believing that such a marriage is valid. So, the children of the marriage that is established based on some suspicions are traced back to their real father. The religion of children must be Islam since they should always be affiliated to the parent who has the best religion whether the marriage is valid or not.
For more benefit on the religion of children when one of the parents is Muslim and another is not, please refer to Fataawa 81957, 86359, and 91184.
If this woman were a Jew or Christian, then the marriage would have been valid provided she was chaste. Thereupon, it would have been lawful for him to continue the marriage contract. In any case, the children are traced back to their father and they follow his religion; i.e., Islam since it is the best religion and Allah will not accept any other religion. Allah Says (what means): {This day I have perfected for you your religion and completed My favor upon you and have approved for you Islam as religion.} [Quran 5:3]
In fact, only the small children who have not yet reached puberty or insane children (even if grown up) have to be trained to follow the best religion. But after puberty their religion is what they choose. If one's father is Muslim, the child is Muslim before his puberty but after puberty, if he chooses any other religion he is considered as an apostate. May Allah Protects us from that. Therefore, the father is obliged to teach his children Islam to protect them from Hellfire and to fulfill his duty as well.
Allah knows best.