The effect of the environment on fitness in natural populations is a fundamental question in evolutionary biology. However, experimental manipulations of both environment and phenotype at the same time are rare. Thus, the relative importance of the competitive environment versus intrinsic organismal performance in shaping the location, height, and fluidity of fitness peaks and valleys remains largely unknown. Here, we experimentally tested the effect of competitor frequency on the complex fitness landscape driving adaptive radiation of a generalist and two trophic specialist pupfishes, a scale-eater and molluscivore, endemic to hypersaline lakes on San Salvador Island (SSI), Bahamas. We manipulated phenotypes, by generating 3407 F4/F5 lab-reared hybrids, and competitive environment, by altering the frequency of rare transgressive hybrids between field enclosures in two independent lake populations. We then tracked hybrid survival and growth rates across these four field enclosures for 3-11 months. In contrast to competitive speciation theory, we found no evidence that the frequency of hybrid phenotypes affected their survival. Instead, we observed a strikingly similar fitness landscape to a previous independent field experiment, each supporting multiple fitness peaks for generalist and molluscivore phenotypes and a large fitness valley isolating the divergent scale-eater phenotype. These features of the fitness landscape were stable across manipulated competitive environments, multivariate trait axes, and spatiotemporal heterogeneity. We suggest that absolute performance constraints and divergent gene regulatory networks shape macroevolutionary (interspecific) fitness landscapes in addition to microevolutionary (intraspecific) competitive dynamics. This interplay between organism and environment underlies static and dynamic features of the adaptive landscape.