Awareness of mild traumatic brain injury and its potential long term effects has increased dramatically in recent years. Currently, there is a lack of computational methods for the evaluation of mild traumatic brain injury (mTBI) from magnetic resonance imaging (MRI). Further, the development of automated analyses has been hindered by the subtle nature of mTBI abnormalities, which appear as low contrast MR regions. In manual estimation a train operator must spend a copious amount of time locating and delineating the lesions. We have developed three innovations to aid in the analysis of mTBI by automatically detecting lesion in MR images. (1) A high level Bayesian network based contextual model, which is able to estimate the progression of lesion over time given basic information about the subject. The generative contextual model is fused with a discriminative visual model that estimates the lesion using 3D features extracted from T2maps. (2) Low level dynamic and static contextual features are developed that take advantage of sequential imaging to improve detection as an iterative post processing step. After the initial estimate by a discriminative classifier, the context features are estimated from the posterior marginal to describe spatio-temporal information about the lesion to the classifier. (3) Multi-Modal features are developed to further improve the visual model. A temporal generative tissue model is developed to model tissue type probabilities within the brain over the course of the injury in the T2map modality. Maximum a posterior edge features are developed to take advantage of the susceptibility weighted imaging (SWI) channel. All of the proposed methods have been evaluated on a temporal mTBI dataset with manually evaluated ground truth and have been shown to outperform the current state-of-the-art approaches.