What does it mean for a neural network to become a “cardinal principal knower”? We trained a multilayer perceptron to compute the successor of the numbers 0-99. N and N+1 were one-hot encoded on the input and output layers, respectively; the hidden layer had 8 units. 80% of the (N, N+1) pairs constituted the training data, the remaining 20% the test data. The mean cosine similarities of the hidden layer representations of the (N, N+1) pairs was 0.77 (0.79) when N was in the training (test) set. The network learned a continuous notion of number: the hidden-layer representations of N and N+1 were comparable whether they did (0.74) or did not (0.78) cross a tens boundary. Thus, the network did not “discover” place-value. Ongoing research is exploring place-value encoding of inputs and outputs, and also structuring of the training data to better reflect the numerical environment of the child.

As children enter elementary school, their understanding of the ordinal structure of numbers transitions from a memorized count list of the first 50-100 numbers to knowing the successor function and understanding the countably infinite. We investigate this developmental change in two neural network models that learn the successor function on the pairs (N, N+1) for N in (0, 98). The first uses a one-hot encoding of the input and output values and corresponds to children memorizing a count list, while the second model uses a place-value encoding and corresponds to children learning the language rules for naming numbers. The place-value model showed a predicted drop in representational similarity across tens boundaries. Analysis of the latent representation shows that counting across a tens boundary can be understood as a vector operation in 2D space, where the numbers with the same tens place are organized in a linearly separable manner, whereas those with the same ones place are grouped together. A curriculum learning simulation shows that, in the expanding numerical environment of the developing child, representations of smaller numbers continue to be sharpened even as larger numbers begin to be learned. These models set the stage for future work using recurrent architectures to move beyond learning the successor function to simulating the counting process more generally, and point towards a deeper understanding of what it means to understand the countably infinite.