In this study, new pyrazole, imidazole, pyrimidine derivatives having imidazo[4,5-b]indol moiety were successfully synthesized, elucidated by spectroscopic techniques, and evaluated as potential antimicrobial agents. The structure-activity relationship was investigated to obtain a better understanding of the relationship between the chemical structure of the synthesized compounds and their corresponding biological activity. Compounds 2b and 3b exhibited potent antibacterial activities against Bacillus subtilis bacteria comparable to that of Ampicillin standard. Structure-activity relationship studies revealed that the presence of withdrawing carbonyl group on 5-position of pyrazole moiety 2b, phenylpyrazole moiety 3b led to an enhancement in the antibacterial activity of pyrazole derivatives. Furthermore, the presence of carbonyl group on 2-position of the pyrimidine ring of compounds 4a, 5a and 6a has a significant effect on their antibacterial activity against Bacillus subtilis. The antifungal studies indicated that compounds 3b, 4b, 7 and 9 have comparable antifungal activity to that of standard Amphotericin B against Candida albicans and Aspergillus flavus fungi.