All perfect praise be to Allaah, The Lord of the worlds. I testify that there is none worthy of worship except Allaah and that Muhammad is His slave and Messenger.
What you mentioned in the question is not correct; rather, anyone who commits a sin, regardless of whether this sin is between him and his Lord or between him and other people, is to be blamed and is subjected to punishment, and it is an obligation to prohibit him from doing evil and to admonish him until he repents to Allaah.
If a person abandons the prayer, then he deserves condemnation more than someone who killed a soul, drank alcohol, committed Zina (adultery or fornication), or stole.
Indeed, Ibn Al-Qayyim quoted the consensus of the scholars on the fact that abandoning the prayer is a greater sin than adultery, theft, consuming alcohol, and killing a soul. This is in terms of the rulings of this worldly life.
If we see someone doing a particular sin, it is an obligation to forbid him, advise him, and order him to repent regardless of whether this sin is related to one of the rights of Allaah or related to the rights of the people.
As regards the Hereafter, the matter of all slaves is in the Hands of Allaah, and He forgives whomever He wills and punishes whomever He wills, but we hope that Allaah will forgive the good doer, and we fear the punishment for the evil doer. However, it is not permissible for us to authoritatively assert that a specific person will definitely be in Paradise or in Hellfire unless this was determined by revelation.
In brief, if someone commits a forbidden matter, we say to him: You have committed a forbidden matter and subjected yourself to the wrath of Allaah, but we do not issue a ruling in regard to his fate in the Hereafter as this is a matter that depends on Allaah alone.
Allaah knows best.