All perfect praise be to Allaah, The Lord of the Worlds. I testify that there is none worthy of worship except Allaah, and that Muhammad is His slave and Messenger.
Allaah Says (what means): {So from wherever you go out [for prayer, O Muhammad sallallaahu ‘'alayhi wa sallam], turn your face toward al-Masjid al-Haram, and indeed, it is the truth from your Lord. And Allaah is not unaware of what you do.} [Quran 2:149].
Therefore, whoever enters a town and he is not sure about the direction of Qiblah, he must ask the residents. If two trustworthy Muslims (who perform the prayer and know its direction) amongst them attest on its direction, then he should follow them and he is not permitted to contradict them on the basis of doubt. Also, he is not permitted to exert any efforts in finding its direction. He is only permitted to do so if there is no trustworthy person who can witness on its direction.
Therefore, the Shaykh should have performed the prayer with you in congregation, and if he had any sound evidence that the people were mistaken, he should have explained it to you. However, he was not permitted to cause doubt among the people and refuse to pray with them.
Nowadays, there are modern instruments for showing the direction of Qiblah, so you may rely on them.
We would like to warn you against a group of innovators who endeavour to cause doubt and confusion among the Muslims in their acts of worship and cause division amongst them. They also try to instigate the issue of Qiblah wherever they go.
Hence, you should not listen to their claim as it is better to follow the truth.
Allaah Knows best.