Chapter 11 Conclusions
We have established a topological deep learning (TDL) framework that enables the learning of representations for data supported on topological domains. To this end, we have introduced combinatorial complexes (CCs) as a new topological domain to model and characterize the main components of TDL. Our framework provides a unification for many concepts that may be perceived as separate in the current literature. Specifically, we can reduce most deep learning architectures presented thus far in the literature to particular instances of combinatorial complex neural networks (CCNNs), based on computational operations defined on CCs. Our framework thus provides a platform for a more systematic exploration and comparison of the large space of deep learning protocols on topological spaces.
Limitations. This work has laid out the foundations of a novel TDL framework. While TDL has great potential, similar to other novel learning frameworks, there are limitations, many of which are still not well-understood. Specifically, some known limitations involve:
- Computational complexity: The primary challenge for moving from graphs to richer topological domains is the combinatorial increase in the complexity to store and process data defined on such domains. Training a TDL network can be a computationally intensive task, requiring careful consideration of neighborhood functions and generalization performance. TDL networks also require a large amount of memory, especially when working with a large number of matrices during network construction. The topology of the network can also increase the computational complexity of training.
- The choice of the neural network architecture: Choosing the appropriate neural network architecture for a given dataset and a given learning task can be challenging. The performance of a TDL network can be highly dependent on the choice of architecture and its hyperparameters.
- Interpretability and explainability: The architecture of a TDL network can make it difficult to interpret the learnt representations and understand how the network is making predictions.
- Limited availability of datasets: TDL networks require topological data, which we have found to be of limited availability. Ad hoc conversions of existing data to include higher-order relations may not always be ideal.
Future work. Aforementioned limitations leave ample room for future studies, making the realization of the full potential of TDL an interesting endeavour. While the flexible definition of CCs, as compared to, e.g., simplicial complexes, already provides some mitigation to the associated computational challenges, improving the scaling of CCNNs even further will require the exploration of sparsification techniques, randomization and other algorithmic improvements. Besides addressing the aforementioned limitations, promising directions not treated within this paper include explorations of directed (Ausiello and Laura 2017), weighted (Battiloro et al. 2023), multilayer (Menichetti, Dall’Asta, and Bianconi 2016), and time-varying dynamic topological domains (Torres et al. 2021; Anwar and Ghosh 2022; Yin et al. 2022). There are also several issues related to the selection of the most appropriate topological domain for a given dataset in the first place, which need further exploration in the future. Additionally, there is a need, but also a research opportunity, to better understand CCNN architectures from a theoretical perspective. This could in turn lead to better architectures. To illustrate this point, consider graph neural networks (GNNs) based on message passing (Gilmer et al. 2017), which have recently been shown to be as powerful as the Weisfeiler–Lehman isomorphism test9. (K. Xu et al. 2018; Maron et al. 2019; Morris et al. 2019). This connection between GNNs and classical graph-based combinatorial invariants has driven theoretical developments of the graph isomorphism problem and has inspired new architectures (K. Xu et al. 2018; Maron et al. 2019; Arvind et al. 2020; Bouritsas et al. 2023). We expect that connecting similar developments will also be important for TDL.
The topological viewpoint we adopt brings about many interesting properties. For example, we are able to model other types of topological inductive biases in our computational architectures, such as the properties that do not change under different discretizations of the underlying domain, e.g., the Euler characteristic that is commonly used to distinguish topological spaces. While isomorphisms are the primary equivalence relation in graph theory, homeomorphisms and topological equivalence are more relevant for data defined on topological spaces, and invariants under homeomorphisms have different machine learning applications10. Homeomorphism equivalence is more relevant in various applications in which domain discretization is an artifact of data processing, and not an intrinsic part of the data (Sharp et al. 2022). Further, homeomorphism equivalence translates to a similarity question between two structures. Indeed, topological data analysis has been extensively utilized towards addressing the problem of similarity between meshes (Dey et al. 2010; Hajij et al. 2018; Rieck, Bock, and Borgwardt 2019). In geometric data processing, neural network architectures that are agnostic to mesh discretization are often desirable and perform better in practice (Sharp et al. 2022). We anticipate that the development of TDL models will open up new avenues to explore topological invariants across topological domains.
References
The Weisfeiler–Lehman isomorphism test (Weisfeiler and Leman 1968) is a widely-used graph isomorphism algorithm that provides a coloring of a graph’s vertices, and this coloring gives a necessary condition for two graphs to be isomorphic.↩︎
Intuitively, two topological spaces are equivalent if one of them can be deformed to the other via a continuous transformation.↩︎