All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad, sallallaahu ‘alayhi wa sallam, is His slave and Messenger.
If the towel with which you dried your body was soiled with impurity, then your body becomes impure by coming into direct contact with this towel because impurity is transferred from one body to another if one of them is wet. Please refer to Fatwa 229372.
Therefore, you have to pour water on the place of your body which you are certain that the impurity has touched. If you only suspect that part of your body came into direct contact with impurity, then the presumption is that it is pure, and if you only suspect that the towel is impure, then the presumption is that it is pure, and therefore, your body is not determined to be impure.
Some scholars are of the view that impurity does not transfer if a moist object comes into direct contact with a dry object and causes it to be slightly moist only. It seems that you are afflicted with obsessive whispers, so it is permissible for you to act according to this opinion in order to avoid hardship.
However, if you are certain that your body has become impure, then it is purified by pouring water on it, and the water that passes over the place of impurity is not considered to be impure unless its qualities change from the impurity.