If you have arrived here, you are probably interested in downloading brain-related data (neural activities, behavioral patterns, etc.). There are two reasons you likely seek such data. A first reason is that you think these data may inform a model of visual system function that you have built or are thinking of building. If so, please read “Data for developing new visual system models” below, and welcome to the team! A second reason is that you have an idea for some new kind of analyses that you would like to try. If so, please go to “Brain data format” (below) and have at it.
Data for developing new visual system models
Welcome! We share your goal of using experiments (aka brain data) to help build better models of the visual system! We have thought a lot about this and we have been working to assemble data from our lab and other labs into a platform that allows your model to fairly draw on the data for testing without the need for a bunch of data wrangling on the side of the model builder. This also allows you, the model builder, to test your model on experiments and associated brain data that we have prepared in this way that you might not have even been thinking about when you arrived at this web site (e.g. response data from V1, or behavioral patterns in humans doing visually-guided tasks). That platform is called Brain-Score, and you can here submit your model(s). Models with the highest brain match (i.e. leading scientific hypotheses) will be prominently displayd on the Brain Score web site.
If you would like to read more about the strategy and philosophy behind this brain benchmarking approach, as well as the technical challenges, please read here (Schrimpf et al., 2020).
Contribute Brain Data and/or Metrics on Brain Data
Building and testing models is not the only way to contribute to the scientific understanding of the mechanisms of natural visual intelligence. Experiments are essential! If you have neural or behavioral data or metrics that you would like to contribute to help drive the development of ventral stream models please get in touch with us to submit data. Similarly, if you have ideas about better ways to compare existing data with models (we call those “Metrics”) then also please get in touch with us.
If you contribute to this community effort, we will help make sure that your work is cited by modelers and we would also love to have you as a co-author on the next Brain Score update paper!
Brain data format (DiCarlo Lab)
In effort to standardize neural and behavioral data we have established a data format we call a Data Assembly. A Data Assembly is a sub-class of xarray DataArray. All behavioral and neural Data Assemblies are handled with the xarray framework. For more information on x-array, please visit: http://xarray.pydata.org.
In general we name each Data Assembly according to the lab in which data was collected, the first/main author who collected/published the data and the year in which the data was collected/published. Sometimes there will be other descriptors such as the stimulus set used to collect the data. For example dicarlo.Rajalingham2018.public means the data was collected in the DiCarlo Lab by Rishi Rajalingham, the data was published in 2018, and the public descriptor notes that the Assembly is open-source.
Each Data Assembly includes the stimulus set used to collect the neural/behavioral data. Further guidance on the structure of Data Assemblies and how to download and work with Data Assemblies can be found on the Brain-Score Github page. In general, a data assembly can be obtained by using a few simple lines of code:
import brainio_collection
assembly =
brainio_collection.get_assembly('dicarlo.MajajHong2015.public')
print(assembly) # data
print(assembly.stimulus_set) # stimuliFor additional help see our Data Assembly tutorial.
Data Assemblies
For an up-to-date list of assemblies use:
brainio_collection.list_assemblies()Note that some of these assemblies are for private evaluation and cannot be downloaded. Below we list the publicly available Data Assemblies which can be loaded by calling their identifier (e.g., dicarlo.MajajHong2015.public) as above. In the future, as more data is collected by our lab and our collaborators it will be packaged into Data Assemblies.
Public Brain Data Assemblies (as of May 2021)
We ask that when publishing any work using the data below that you cite that specific data set. For each public Data Assembly we provide the identifier and a link to the publication. If the work has not been published we will sometimes provide a DOI that includes experimental methodologies.
dicarlo.Rajalingham2018.public Rajalingham et al., 2018
dicarlo.MajajHong2015.public Majaj et al., 2015
dicarlo.MajajHong2015.temporal.public Majaj et al., 2015
movshon.FreemanZiemba2013.public Freeman et al., 2013
Open-Source Published Data (Available for Download, Not in Assembly Format)
Kar et al., 2019 Nature Neuroscience