This paper presents a thorough analysis of potential-based and non-potential-based cohesive zone models (CZMs) under conditions of mixed-mode separation and mixed-mode over-closure. Problems are identified with the well established potential-based Xu-Needleman (XN) model and a number of new potential-based and non-potential-based models are proposed. It is demonstrated that derivation of traction-separation relationships from a potential function can result in non-physical repulsive normal tractions and instantaneous negative incremental energy dissipation under displacement controlled monotonic mixed-mode separation when the work of tangential separation exceeds the work of normal separation. A modified potential-based (MP) model is proposed so that the zone in which repulsive normal tractions occur can be controlled. The MP model also provides an additional benefit of correct penalisation of mixed-mode over-closure, in contrast to the XN model. In order to fully eliminate the problem of repulsive normal tractions a non-potential-based CZM (NP1) is also proposed. This model is shown to provide physically realistic behaviour under conditions of displacement controlled mixed-mode separation and over-closure. Noting that the form of the traction-separation equations differ for mode I and mode II separation for the XN, MP and NP1 models, an additional non-potential-based model (NP2) is proposed so that near mode-independent behaviour can be achieved in displacement controlled separation, while correctly penalising over-closure. Following from the NP2 model, a non-potential-based model in which coupling is based on the separation magnitude is considered (SMC model). In the final part of the paper the performance of each model under traction controlled mixed-mode separation is investigated by numerically inverting the traction-separation equations. Separation paths for the XN model reveal a strong bias toward mode I separation while the NP1 model exhibits a bias towards mode II separation. Interestingly, the NP2 model exhibits a high degree of mode sensitivity under traction controlled conditions, in contrast to its near mode independence under displacement controlled conditions. It is demonstrated that incorrect weighting of the coupling terms in non-potential models can lead to the existence of a singularity under traction controlled conditions. Finally, it is demonstrated that the potential-based models fail to capture a gradual change from mode II to mode I work of separation, as reported experimentally for traction controlled interface separation. In a follow-on Part II companion paper a number of case studies are simulated, demonstrating that the theoretical findings of the present paper have significant implications for the finite element prediction of interface debonding. © 2013 Elsevier Ltd.