All perfect praise be to Allah, the Lord of the worlds. I testify that there is none worthy of worship except Allah and that Muhammad, sallallahu ‘alayhi wa sallam, is His slave and Messenger.
May Allah reward you for your keenness on raising your children to be good and righteous. We ask Him to bless your efforts with success, and we advise you to supplicate Allah for them as much as you can, as the supplications of the parents for their children are accepted. Abu Hurayrah, may Allah be pleased with him, narrated that the Prophet, sallallahu ‘alayhi wa sallam, said, "There are three supplications that will undoubtedly be answered: the supplication of one who has been wronged, the supplication of the traveler, and the supplication of the parent for his child." [Ibn Maajah - Al-Albaani graded it hasan (good)]
You should know that the grandmother is held in the same status as the mother and the grandfather is held in the same status as the father in terms of their right upon their grandchildren to be kind and dutiful towards them. Ibn Al-Munthir said, "Grandfathers are fathers, and grandmothers are mothers. And a Muslim should not participate in jihaad (fighting in the cause of Allah) without their consent. I do not know of any proof for the need to ask for the consent of other than them, whether it be the siblings or any other relatives."
Ibn Hazm said, "The scholars unanimously agreed that dutifulness towards the parents is a religious obligation, and they also unanimously agreed that dutifulness towards the grandparent is an obligation." [Al-Ijmaaʻ]
You should know that one of the greatest manifestations of dutifulness towards them is striving for the sake of guiding them and reforming their religion. You can accomplish this objective firstly by supplicating Allah, the Exalted, to bless them with guidance and then by giving them advice kindly and wisely or by seeking the help of someone whom you believe they would most likely listen to.
It is impermissible for you, under any circumstances, to sever the ties with them completely. So uphold the ties with them by whatever possible means available, such as visits or otherwise. Do not prevent your children from contacting them, because this would be considered cooperation in severing the ties of kinship. Rather, you should try to prevent what may make them have a bad influence on them. If refraining from visiting them achieves this purpose, then there is no harm in not visiting them provided that you keep ties with them by other means. Note that the ways to keep ties of kinship are determined by the customs and traditions of the community. For more benefit, please refer to fatwa 143169.
Allah knows best.