Probing the effects of object category learning on the macaque inferior temporal cortex

Like humans, adult non-human primates can learn to categorize visual objects. Much prior work shows that individual neurons in the inferior temporal (IT) cortex, which is critical for visual object discrimination, modestly increase their selectivity to objects from learned categories. While the field now has relatively accurate models of IT responses, we do not yet have a similar understanding of adult IT plasticity or its role in behavioral performance gains (“learning”). To begin to address this, we measured changes in IT due to object category learning and asked how those quantitatively relate to behavior. We performed multielectrode recordings in two groups of macaques (3 monkeys/group), while monkeys viewed naturalistic images (8 categories, 80 images/category, 100 ms). Prior to recording, one group (naive) was trained to fixate passively on images; the other group (trained) also learned to discriminate multiple object categories via operant conditioning. We randomly sampled 58 reliable, visually responsive sites from each monkey to construct two pools of IT activity (178 sites per pool/group). Consistent with previous studies, a category-based representational similarity analysis revealed a small (13.81%) but significant difference in representation between the trained and naive IT population. Not surprisingly, this representational shift corresponded to a small, statistically significant increase (~ +10%) in IT-based linear decoding accuracy of learned categories. Notably, this inferred increase in the linearly separable category information in the trained IT was much smaller than improvements observed in the monkeys’ behavior (~ +40% accuracy gain). To probe the driving factors underlying these incommensurate changes across IT and behavior, we cast our learning paradigm as an extension of contemporary artificial neural networks (ANNs), the leading models of the ventral stream. We observed that IT layers in various task-optimized ANNs (different architectures, pre-training objectives, category learning schemes) showed monkey-IT-like increases in category information after training. Interestingly, akin to IT, where trained IT decodes were more consistent with image-level behavioral patterns than naive IT decodes, specific ANN-ITs were more aligned with monkey behavior after training. In sum, our findings indicate that category learning produces modest changes in the IT cortex, enhancing category information readout. We developed a computational framework to simulate these transformations, enabling us to formulate testable hypotheses about other representational reconfigurations induced by category learning.