All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad, sallallaahu ‘alayhi wa sallam, is His Slave and Messenger.
The Wali (guardian) is a requisite for the validity of the marriage contract, and this is the preponderant opinion of the majority of the scholars of Fiqh, except for the Hanafi scholars. This has previously been clarified in Fatwa 86348. In order to know the conditions of a valid marriage in general, see Fatwa 83629.
As you have read, the disbeliever is not suitable to be a guardian for a female Muslim, as Allaah The Almighty Says (what means): {and never will Allaah give the disbelievers over the believers a way [to overcome them].}[Quran 4:141]
Similarly, it is impermissible for a woman to undertake the contract of marriage by herself.
If a woman has no Muslim guardian, a Muslim judge should give her in marriage, due to the statement of the Prophet, sallallaahu ‘alayhi wa sallam: "The ruler is the guardian of the one who has no guardian." [Ahmad] If no Muslim judge is available, the woman then should authorize a man from among the Muslims to undertake the contract of her marriage. For further information, see Fatwa 88488.
Subsequently, if your marriage was done without a guardian, it is an invalid marriage. So, if you wish to maintain the marriage between both of you, you should then renew it according to the right manner, after the end of one menses.
Besides, if you have got children, they are traced back to their father; and there is no difference of opinion (among the jurists) in this regard. So, there is no need to worry.