In this study, we explore the possibility of using the information-theoretic concept of mutual information, between the output signal and the regressor of an ARX system model as a criterion to effectively select the most informative data-sets from a collection of experimental records. We derive an expression connecting the mutual information to the signal properties to help with our analysis. We then use the expression to check whether using mutual information as a design criterion to synthesize the input signal leads to any meaningful connections to identifiability. Our findings indicate that mutual information does not serve as a suitable criterion for experiment design.