Increasingly, cognitive scientists have demonstrated interest inapplying tools from deep learning. One use for deep learning isin language acquisition where it is useful to know if a linguisticphenomenon can be learned through domain-general means.To assess whether unsupervised deep learning is appropriate,we first pose a smaller question: Can unsupervised neural net-works apply linguistic rules productively, using them in novelsituations? We draw from the literature on determiner/nounproductivity by training an unsupervised, autoencoder networkmeasuring its ability to combine nouns with determiners. Oursimple autoencoder creates combinations it has not previouslyencountered and produces a degree of overlap matching adults.While this preliminary work does not provide conclusive evi-dence for productivity, it warrants further investigation withmore complex models. Further, this work helps lay the foun-dations for future collaboration between the deep learning andcognitive science communities.