All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
First of all, we congratulate you upon your repentance, and may Allaah reward you for willing to remain steadfast on the Path of Allaah, and we ask Him to prevent you from straying in the future.
We did not clearly understand you, as on one hand you say that you are married to this woman and on the other hand you say that the children are a result of an illegal relationship with her. Anyway, if they are a result of an illegal relationship, they should not be traced back to you, and you do not have any responsibility towards them. They should be traced back to their mother and they are considered like her in regard to their religion.
For more benefit, please refer to Fatwa 89082.
However, if there is any reason for tracing them back to you, like if they are resulted from a correct marriage, or a marriage which you believed to be correct, then they should be considered as Muslims because they should be traced back to you in this case. Hence, you should not leave them with their mother and you are obliged to do your best to take the children away from her using any means without causing any harm to yourself. If you are unable to do so, then you should establish a strong relationship with them as much as you can. For more benefit, please refer to Fataawa 89418 and 81957.
Since the matter is a little bit ambiguous, then you should contact the scholars in your country and clarify to them your case so that they would issue a Fatwa to you accordingly.
Finally, it should be noted that there are many temptations for a Muslim in residing in a non-Muslim country. Therefore, if there is no necessity or dire need for him to reside there, he should migrate to a Muslim country where he would feel safe about his religion.