Face image modification is a variant of the image-to-image translation task where we modify features of a face image to evoke given target attributes, while preserving the identity of the pictured person. Generative Adversarial Networks (GANs) using an encoder-decoder architecture have been widely used to modify both discrete and continuous face attributes, with a few different architectures designed to address the challenge of preserving identity through the modification. We propose a novel GAN architecture that introduces gated skip connections in the generator's decoder for this task. Our model enables high fidelity (512x512) modification with minimal changes to irrelevant facial regions, while using fewer parameters than existing approaches. We demonstrate the model on discrete CelebA attributes, continuous facial Action Unit labels, and perceived social impression traits such as "attractive", "kind", and "trustworthy". Our model is also able to selectively visualize the modified face features, allowing us to extract plausible visual explanations for face attributes, including, for the first time, social impression traits. An experiment with human raters validates that our model can effectively alter a face's social impressions.