Gene transcription is regulated in response to environmental changes and developmental cues. In mammalian cells subjected to stress conditions such as heat shock, transcription of most protein-coding genes decreases, while the transcription of heat shock protein genes increases. Repression involves direct binding to RNA polymerase II (Pol II) of certain noncoding RNAs (ncRNAs) that are upregulated upon heat shock. Another class of ncRNAs is also upregulated and binds to Pol II but does not inhibit transcription. Incorporation of repressive ncRNAs into pre-initiation complexes prevents transcription initiation, while non-repressive ncRNAs are displaced from Pol II by TFIIF. Here, we present cryo-electron microscopy reconstructions of human Pol II in complex with six different ncRNAs from mouse and human. Our structures show that both repressive and non-repressive ncRNAs bind to a conserved binding site within the cleft of Pol II. The site, which is also shared with a previously characterized yeast aptamer, is close to the active center and, thus, in an ideal position to regulate transcription. Importantly, additional RNA elements extend flexibly beyond the docking site. We propose that the differences concerning the repressive activity of the ncRNAs analyzed must be due to the distinct character of these more unstructured, flexible segments of the RNA that emanate from the cleft.