Publications
C. Maffei, G. Girard, K.G. Schilling, D.B. Aydogan, N. Adluru, A. Zhylka, Y. Wu, M. Mancini, A. Hamamci, A. Sarica, A. Teillac, S.H. Baete, D. Karimi, F.-C. Yeh, M.E. Yildiz, A. Gholipour, Y. Bihan-Poudec, B. Hiba, A. Quattrone, A. Quattrone, P.-T. Yap, A. de Luca, J. Pluim, A. Leemans, V. Prabhakaran, B.B. Bendlin, A.L. Alexander, B.A. Landman, E.J. Canales-Rodríguez, M. Barakovic, J. Rafael-Patino, T. Yu, G. Rensonnet, S. Schiavi, A. Daducci, M. Pizzolato, E. Fischi-Gomez, J.-P. Thiran, G. Dai, G. Grisot, N. Lazovski, S. Puch, M. Ramos, P. Rodrigues, V. Prchkovska, R. Jones, J. Lehman, S.N. Haber, A. Yendiki, Insights from the IronTract challenge: optimal methods for mapping brain pathways from multi-shell diffusion MRI, NeuroImage, 257:119327, 2022.
C. Maffei, G. Girard, K.G. Schilling, D.B. Aydogan, N. Adluru, A. Zhylka, Y. Wu, M. Mancini, A. Hamamci, A. Sarica, D. Karimi, F.-C. Yeh, M.E. Yildiz, A. Gholipour, A. Quattrone, A. Quattrone, P.-T. Yap, A. de Luca, J. Pluim, A. Leemans, V. Prabhakaran, B.B. Bendlin, A.L. Alexander, B.A. Landman, E.J. Canales-Rodríguez, M. Barakovic, J. Rafael-Patino, T. Yu, G. Rensonnet, S. Schiavi, A. Daducci, M. Pizzolato, E. Fischi-Gomez, J.-P. Thiran, G. Dai, G. Grisot, N. Lazovski, S. Puch, M. Ramos, P. Rodrigues, V. Prchkovska, R. Jones, J. Lehman, S. Haber, A. Yendiki. New insights from the IronTract challenge: Simple post-processing enhances the accuracy of diffusion tractography, Proc. Intl. Soc. Mag. Res. Med., 2021 (oral presentation, Magna Cum Laude).
C. Maffei, G. Girard, K. G. Schilling, N. Adluru, D. B. Aydogan, A. Hamamci, F.-C. Yeh, M. Mancini, Y. Wu, A. Sarica, A. Teillac, S. H. Baete, D. Karimi, Y.-C. Lin, F. Boada, N. Richard, B. Hiba, A. Quattrone, Y. Hong, D. Shen, P.-T. Yap, T. Boshkovski, J. S. W. Campbell, N. Stikov, G. B. Pike, B. B. Bendlin, A. L. Alexander, V. Prabhakaran, A. Anderson, B. A. Landman, E. J. Z. Canales-Rodríguez, M. Barakovic, J. Rafael-Patino, T. Yu, G. Rensonnet, S. Schiavi, A. Daducci, M. Pizzolato, E. Fischi-Gomez, J.-P. Thiran, G. Dai, G. Grisot, N. Lazovski, A. Puente, M. Rowe, I. Sanchez, V. Prchkovska, R. Jones, J. Lehman, S. Haber, A. Yendiki. The IronTract challenge: Validation and optimal tractography methods for the HCP diffusion acquisition scheme, Proc. Intl. Soc. Mag. Res. Med., 2020 (oral presentation, Magna Cum Laude).
Rationale
Diffusion MRI tractography is a technique for reconstructing the white-matter axon bundles that form the circuitry of the brain. Its advantage is that it relies on a relatively short in vivo brain scan to extract information that in the past was only accessible by time-consuming post mortem dissections. Its drawback is that it uses indirect measurements on the arrangement of these axon bundles, based on how water molecules diffuse in and around them, and hence its accuracy is limited by the inherent ambiguities in these measurements.
Many tractography algorithms have been proposed in the literature, and some have made it into publicly available software tools. The objective assessment of the accuracy of these algorithms would help identify possible avenues for their improvement. This would be beneficial to all applications where tractography is used, including studies of neurological and psychiatric conditions, as well as brain development and aging in healthy subjects. Furthermore, several initiatives are currently generating thousands of high-quality diffusion MRI data sets. As these data are being shared with the entire neuroimaging community, it is of broad interest to identify the optimal methods for analyzing them.
Data
The IronTract challenge uses ex vivo diffusion MRI and anatomic tracing data from the same macaque brains:
- Diffusion MRI: Fixed brains were scanned in a small-bore 4.7T Bruker scanner with a 3D echo-planar imaging sequence (isotropic resolution 0.7mm, TR=750ms, TE=43 ms, 𝛿=15ms, Δ=19ms). Diffusion sensitization was applied in 515 directions arranged on a uniform lattice in q-space, within a sphere with maximum b=40,000s/mm2 (roughly equivalent to 10,000 in vivo). From these data, which were collected on a Cartesian grid in q-space, we can generate data arranged on q-shells by using a fast implementation of the non-uniform Fourier transform.
- Anatomic tracing: Bidirectional tracers were injected in the frontal cortex. Both the injection sites and the brains differ between the cases used for training and validation in this challenge. In each case, however, the tracer data come from the same brain as the diffusion MRI data.
Evaluation metrics
The area under the receiver-operating characteristic (ROC) curve will be used to rank challenge entries. Participants will upload tractography volumes generated at different operating points of their method (obtained, e.g., by varying the bending angle threshold for deterministic methods or the probability threshold for probabilistic methods). The true positive rate (TPR) and false positive rate (FPR) will be evaluated for each uploaded volume by comparing it to the tracing, which will be unseen by participants.
Entries will be ranked in two ways to produce two winners:
- Overall winner: For this ranking, participants will be allowed to use whichever q-space sampling scheme (single-shell, multi-shell, Cartesian grid, etc.) works best for their method. The goal is to find the tractography method that achieves the highest accuracy possible today, regardless of acquisition scheme.
- HCP winner: For this ranking, participants will be restricted to the acquisition scheme that is deployed by both lifespan studies and disease studies of the Human Connectome Project (HCP). This scheme comprises two q-shells with b=1500 and 3000 (adjusted here to account for the 4x factor required to achieve the same diffusion contrast in a fixed brain). The goal is to find the method that is best suited for analyzing the 6500+ data sets that are being collected across the lifespan and disease HCP studies, and that will soon be available publicly.
Why do we need another challenge?
Previous tractography challenges (ISMRM 2015, ISBI 2018) offered invaluable opportunities for developers to assess the performance of their methods on a common testbed, and generated important insights into the relative merits of different approaches. In addressing the questions that they set out to answer, these pioneering efforts have given rise to new ones.
First, previous challenges have relied on data that were either simulated or acquired with a single, low b-value. (Two b-values have only been made available on phantom and not brain data.) This precludes the use of many state-of-the-art reconstruction methods that require higher or multiple b-values. It is also not representative of the state-of-the-art, large data sets produced by initiatives such as the HCP. We provide data with a full Cartesian sampling of q-space on 515 points up to a b-value of 40,000, which we can resample on arbitrary q-shells using a fast non-uniform Fourier transform. This offers participants substantial flexibility, as it allows the use of reconstruction methods that require Cartesian, multi-shell, or single-shell data.
Second, previous challenges asked participants to provide a single output from each tractography method (at a single threshold of their choice). Then methods were ranked separately by sensitivity or by specificity. Unsurprisingly, different methods were found to have both different sensitivity and different specificity. While this provides invaluable insight into the default operating points (i.e., default thresholds) of different methods, it does not facilitate side-by-side comparisons. That is, it does not tell us which method would achieve the highest sensitivity, if they were adjusted to operate at the same level of specificity. In this challenge, we ask participants to vary the thresholds of their methods so as to explore fully the specificity/sensitivity trade-off.
This challenge involves a side-by-side comparison of tractography and tracer injections in the same brains. The injection sites are in areas of frontal cortex that involve a complex system of small axon bundles that travel within multiple large white-matter highways, following detailed topographies that we have shown to be homologous across species. This task differs from reconstructing large highways such as the corticospinal tract, cingulum bundle, etc.