Non-invasive strategies that can identify oral malignant and dysplastic oral potentially-malignant lesions (OPML) are necessary in cancer screening and long-term surveillance. Optical coherence tomography (OCT) can be a rapid, real time and non-invasive imaging method for frequent patient surveillance. Here, we report the validation of a portable, robust OCT device in 232 patients (lesions: 347) in different clinical settings. The device deployed with algorithm-based automated diagnosis, showed efficacy in delineation of oral benign and normal (n = 151), OPML (n = 121), and malignant lesions (n = 75) in community and tertiary care settings. This study showed that OCT images analyzed by automated image processing algorithm could distinguish the dysplastic-OPML and malignant lesions with a sensitivity of 95% and 93%, respectively. Furthermore, we explored the ability of multiple (n = 14) artificial neural network (ANN) based feature extraction techniques for delineation high grade-OPML (moderate/severe dysplasia). The support vector machine (SVM) model built over ANN, delineated high-grade dysplasia with sensitivity of 83%, which in turn, can be employed to triage patients for tertiary care. The study provides evidence towards the utility of the robust and low-cost OCT instrument as a point-of-care device in resource-constrained settings and the potential clinical application of device in screening and surveillance of oral cancer.