The human cerebral cortex is one of the most evolved regions of the brain, responsible for most higher-order neural functions. Since nerve cells (together with synapses) are the processing units underlying cortical physiology and morphology, we studied how the human neocortex is composed regarding the number of cells as a function of sex and age. We used the isotropic fractionator for cell quantification of immunocytochemically labeled nuclei from the cerebral cortex donated by 43 cognitively healthy subjects aged 25-87 years old. In addition to previously reported sexual dimorphism in the medial temporal lobe, we found more neurons in the occipital lobe of men, higher neuronal density in womens frontal lobe, but no sex differences in the number and density of cells in the other lobes and the whole neocortex. On average, the neocortex has ~10.2 billion neurons, 34% in the frontal lobe and the remaining 66% uniformly distributed among the other 3 lobes. Along typical aging, there is a loss of non-neuronal cells in the frontal lobe and the preservation of the number of neurons in the cortex. Our study made possible to determine the different degrees of modulation that sex and age evoke on cortical cellularity.