Interview with Paula Feldman: generating 3d models of blood vessels

In their work VesselVAE: Recursive Variational Autoencoders for 3D Blood Vessel Synthesis, Paula Feldman and colleagues present a data-driven generative framework for synthesizing blood vessel 3D geometry. We asked Paula about this work, their methodology, and why this is such an interesting area for study.

What is the topic of the research in your paper and why is this an interesting area for study?

A few years ago, various image synthesis methods gained a lot of popularity, leading to the use of synthesis models in several domains including the proliferation of deep fakes across the internet. More recently, DALL-E and stable diffusion techniques have also captured a massive audience. The potential applications of these synthesis techniques in medical imaging have not been overlooked. As a result, several algorithms have been developed to synthesize realistic-looking medical images for diverse medical purposes.

The subject of the paper centers around blood vessels, as accurate 3D models of blood vessels are becoming increasingly essential in the fields of medicine and science. However, reconstructing these thin features with precision from medical images presents a considerable challenge. The process of manually editing vessel geometry is not only tedious but also error-prone, necessitating expert medical knowledge. Consequently, the scarcity of curated datasets is understandable. Several methods have been developed to effectively synthesize blood vessel geometry in response to this challenge. These approaches aim to provide reliable and accurate representations of blood vessels to meet the growing demand for accurate 3D models in medical and scientific applications.

Could you tell us about some of the main challenges in this area of research?

As previously discussed, the scarcity of curated datasets poses a significant challenge and these datasets are also needed for training synthesis models. This scarcity makes it difficult to obtain a diverse and comprehensive set of samples necessary for successful training. Also, because of confidentiality issues and institutional policies, most datasets are not published for public use, which poses a greater challenge for labs without access to the medical facilities to generate their own dataset.

Moreover, the process of training synthesis models presents additional hurdles, including the considerable computational power required and the inherent instability of the training process. The complexity of these models demands significant computing resources, which can be costly and time-consuming. Furthermore, achieving stability during training is often elusive and models may suffer from convergence issues or produce inconsistent results.

Example of generated blood vessel structures.

Could you explain your methodology and how you trained the model?

In this study, we utilized a recursive neural network (RvNN), a specialized architecture of an artificial neural network capable of processing structured data like trees or graphs. By recursively applying the same neural network architecture on nested substructures, the RvNN efficiently captures hierarchical relationships and dependencies present in such data.

To leverage the tree-like structure of blood vessels effectively, we designed an RvNN that could encode and decode this tree structure recursively. This approach allowed us to understand better and model the complex branching patterns and connections within the blood vessels.

For the training phase, we used an open-access dataset containing 700 segments of blood vessels. To facilitate the training process we parametrized the vessels as trees, where each node within the tree possessed parent and/or child nodes, as well as an attribute vector encompassing the node’s coordinates and radius. This parameterization helped represent the intricate structural information of the blood vessels in a format suitable for RNN-based learning.

How did you evaluate the model and what were the results?

Our evaluation process encompassed both quantitative and qualitative assessments. For the quantitative evaluation, we computed a range of metrics for both the vessels generated by our model and the vessels in the dataset. To gain insights into the similarity between the two sets, we rendered histograms for these metrics and calculated the cosine similarity. The results of the quantitative evaluation were highly promising, with a cosine similarity above 0.95 observed for all metrics, indicating a strong resemblance between the generated vessels and the ones in the dataset.

In addition to the quantitative analysis, we conducted a qualitative evaluation through visual comparison with state-of-the-art methods. These alternative methods produce vessels without bifurcations or exhibit straight segments without curvature, making a direct quantitative comparison unfeasible. Nevertheless, the visual assessment proved to be revealing, clearly demonstrating the differences between our model’s outputs and those of existing methods.

Our model demonstrated success in generating blood vessel structures that closely resemble the dataset vessels, as evidenced by the high similarity scores in the quantitative evaluation. Furthermore, the qualitative evaluation showed our method can generate vessels with both varying radii and multiple branches.

What further work are you planning in this area?

We’re interested in incorporating a constraint that allows users to preselect vessel features or determine the type of vessel they want to generate. Additionally, we aim to enhance the system by evaluating the generated vessels through a downstream task, such as flow estimation.

Find out more

• The paper: VesselVAE: Recursive Variational Autoencoders for 3D Blood Vessel Synthesis, Paula Feldman, Miguel Fainstein, Viviana Siless, Claudio Delrieux, Emmanuel Iarussi, Medical Image Computing and Computer Assisted Intervention – MICCAI 2023. Read in full on arXiv.
• Repository, including live demo, code, data

About Paula

Paula Feldman is a Biomedical Engineer from Universidad Nacional de Tucumán, and is currently a PhD candidate at Universidad Torcuato Di Tella with a scholarship from the National Research Council (CONICET).

The AI Around the World series is supported through a donation from the Mohamed bin Zayed University of Artificial Intelligence (MBZUAI). AIhub retains editorial freedom in selecting and preparing the content.

Lucy Smith is Senior Managing Editor for AIhub.

Lucy Smith is Senior Managing Editor for AIhub.