All perfect praise be to Allaah, The Lord of the worlds. I testify that there is none worthy of worship except Allaah and that Muhammad, sallallaahu ʻalayhi wa sallam, is His slave and Messenger.
The ruling on the removal of the hair growing on the cheek and chin is the same. Al-Qaamoos Al-Muheet reads, “Lihyah (beard) means the hair growing on the cheeks and chin.” [p. 1330]
Ibn Hajar said, “Liha, plural of lihyah, which is the hair that grows on the cheeks and chin...” [Fat-h Al-Baari, 10/350]
The Fatwa of the Permanent Committee for Islamic Research and Fataawa reads, “It is impermissible to shave the sides of the face because they are part of the beard ...”
Ibn Baaz was asked, “Is it permissible for men to pluck facial hair, such as hair on the cheeks and above the eyebrows? Please advise us, may Allaah reward you with the best!” He answered:
“It is not permissible for men to pluck any hair from the cheeks because it is considered part of the beard. Neither men nor women are permitted to remove hair from their eyebrows. The Messenger of Allaah, sallallaahu ʻalayhi wa sallam, cursed those who remove hair from the eyebrows and those who have the hair of their eyebrows removed. Nams (removing hair from the eyebrows), thus, is neither permissible for men nor for women. Similarly, the hair growing on the cheeks is considered part of the beard, and it is not permissible for men to remove it...” [Majmoo’ Fataawa Ibn Baaz, 29/39]
Some scholars, such as Imaam Ahmad and some of the Maaliki scholars, held a less stringent view regarding the hair growing on the sides of the face; they held that it is allowable to remove from it. Ash-Shabeehi wrote in Al-Fajr As-Saati’ ʻala As-Saheeh Al-Jaami’ - Chapter: 'Growing the Beard':
“The lihyah means the hair that grows on the two sides of the face and chin; it is obligatory to let this hair grow without shaving or trimming it ... Abu ʻAbdullaah Al-Abiyy said, 'Allaah, The Exalted, adorned men with beards; and since it is a source of adornment, it should be trimmed by removing from its length and width, and the limit is to trim what is more than the fist's length, as Ibn ʻUmar used to do. This is for the men whose beards grow large, so they take from its length and width to improve its appearance. Verily, Allaah is Beautiful and He loves beauty. As for the hair that grows on the cheeks, Shaykh Abu Al-Hasan Al-Muntasir used to let it grow and did not remove it. Other scholars of his level, though, used to remove it. This scholarly view was chosen by our Shaykh.'”
An-Nafraawi wrote, “As for the hair growing on the cheek, Ibn ʻArafah held that it is permissible for the Muslim man to remove it...” [Sharh Al-Risaalah]
In any case, the questioner stated that his beard mostly grows on the cheeks (with much less growing on the bottom of the face) and looks strange and unnatural. If this makes his face look abnormal and it harms him, then he may trim it to the extent that wards off the harm.
Allaah knows best.