A compact protein with a size of <1,000 amino acids, the CRISPR-associated protein CasX is a fundamentally distinct RNA-guided nuclease when compared to Cas9 and Cas12a. Although it can induce RNA-guided genome editing in mammalian cells, the activity of CasX is less robust than that of the widely used S. pyogenes Cas9. Here, we show that structural features of two CasX homologs and their guide RNAs affect the R-loop complex assembly and DNA cleavage activity. Cryo-EM-based structural engineering of either the CasX protein or the guide RNA produced two new CasX genome editors (DpbCasX-R3-v2 and PlmCasX-R1-v2) with significantly improved DNA manipulation efficacy. These results advance both the mechanistic understanding of CasX and its application as a genome-editing tool.