Enhancers harbor instructions encoded for the interactions between cis-elements and transcription factors to orchestrate lineage specific gene programs. Here we developed a modified method for chromosome conformation capture (3C), named MID Hi-C, to reveal how in mouse embryonic stem cells differential cooperation of enhancers and the chromatin remodeler BAF, as instructed by the underlying transcription factor motifs, modulate enhancer-promoter communication. We show that BAF-dependent enhancers permit genomic interactions beyond enhancer boundaries. BAF-dependent enhancers do not dictate genomic interactions within enhancer-promoter loop domains but rather act to instruct remote enhancer-promoter communication. In contrast, BAF-independent enhancers interact with promoter regions within tightly insulated enhancer-promoter loop domains that are marked by promoter and enhancer boundary elements. In addition, enhancer activeness modulated by BAF enforces compartment segregation. Based on these observations, we propose that enhancer cis elements instruct with great precision BAF-induced enhancer-promoter communication and compartmental segregation. We also characterize the gene programs and enhancer landscapes that instructed the cell fate bifurcation in the extraembryonic endoderm lineage orchestrated by lineage specific transcription factors. The simplicity of this developmental system will be valuable for dissecting how enhancer cis elements exploit the 3D genome principles to inscribe proper cell differentiation.