Speaker Series Archive - From Cells to Galaxies

Radio Astronomy 101 and Medical Imaging 101: We began with introductory technical sessions to build a foundation for common language and understanding.

Following the introductory cross discipline sessions, the next four sessions will present joint perspectives from Medical Imaging and Radio Astronomy Imaging researchers on the following topics:

Image Reconstruction Methods: Algorithms for image reconstruction from raw data
Inverse Imaging Techniques: Algorithms and techniques for calibration and data/image domain transformation
Visualization: Tools used for data and image visualization in RA and MI
Intro to Software Systems: General introduction of the capabilities of the software systems used in RA and MI

Common problems/Cross pollination: The final virtual gathering identifies areas of collaboration and next steps for the face to face meeting.

Neb Duric
University of Rochester Medical School

While radio astronomy and medical radiology occupy opposite ends of the imaging scale, they share some remarkable similarities. First, both rely on the detection and processing of digitized (raw) signals. Secondly, both use complex reconstruction algorithms to form images. Thirdly, the systems studied in radio astronomy and medicine are of comparable size when measured in terms of resolving power (beam width). Finally, both disciplines face similar challenges with computational costs, management of large data sets and the need for automated image interpretation. Despite these similarities, the evolution of imaging science in the two disciplines has proceeded on separate, non-intersecting paths. Thus, the respective innovations in imaging have remained largely separated. The purpose of this presentation is to provide an overview of the similarities and differences in imaging methods in order to kick off the meeting whose goal is to identify challenges and opportunities for collaborations between astronomers and medical imagers. The ultimate goal of this meeting is to (i) expose innovations across the two disciplines, (ii) identify areas of overlap and (iii) put into motion practical applications that can further advance each field through the formation of inter-disciplinary collaborative groups.

Video Recording

Urvashi Rau
NRAO

Image formation in radio astronomy and medical imaging have many interesting parallels in terms of the mathematical structure of the systems of equations being solved. However, the two fields have evolved in somewhat distinct ways. For example, radio interferometry began with sparse measurements in Fourier space and is moving in the direction of larger amounts of data to maximize image sensitivity and Fourier space coverage. Medical imaging on the other hand, has always had full control of the sampling of Fourier space, but due to practical constraints such as patient movement and radiation exposure has been evolving in the direction of sparser measurements. Reconstruction algorithms in both fields have evolved accordingly and range from greedy algorithms based on physical interpretations of measurement equations to formal compressed sensing formulations.

This talk will give an overview of image formation in radio astronomy, arranged as a direct counterpart to the medical imaging talk in today's session, and aimed at setting the stage for a clear illustration of the similarities and differences between the fields. The core content of the talk will cover measurement processes in radio astronomy and interferometry, how this translates to a sampling in Fourier space, and the structure and features of the resulting inverse problem. Along the way, I will include comments about the evolution of observational strategies and reconstruction algorithms in the field, directions of future evolution both algorithmically and in data volume, and aspects of the reconstruction that affect the accuracy and reliability of the final result as viewed by a practicing astronomer.

Video Recording

Daniel Sodickson
NYU Langone Health

The fundamental human urge to extend our vision has resulted in the notably distinct but at root highly connected fields of astronomy and medical imaging. Whereas astronomy sets its sights on outer space, medical imaging delves into inner space. As a direct counterpoint to the other talk in today's session, this presentation will introduce some of the key means by which medical imagers gain access to the insides of things. I will first survey the history of medical imaging broadly defined, illustrating some of the fundamental principles of operation of the key cross-sectional imaging modalities used in modern medicine: namely, CT, PET, Ultrasound, and MRI. There is a particular kinship between radio astronomy and MRI, which exploit a shared area of the electromagnetic spectrum, operate in analogous signal spaces, and have a common structure to their inverse problems. I will therefore proceed to focus in on MRI, explaining key principles from a perspective that I hope will resonate with radio astronomers. I will introduce MRI hardware in three simple steps, illustrate the structure and information content of the MR signal, summarize approaches to image formation that will be elaborated upon further in subsequent talks, and explore some of the limits of performance of MR methods. Then, in the end, I will briefly zoom back out to consider the future of medical imaging, assessing how our access to inner space may change with the advent of artificial intelligence and other surprising new technologies and transforms.

Video Recording

Kazunori Akiyama
NRAO, MIT Haystack Observatory

The power of imaging is commonly fundamental to radio astronomy and medical field. In both fields, the imaging process is formulated as underdetermined problems to reconstruct an image from measurements in its Fourier space, which are incompletely sampled for different reasons. The imaging algorithm is designed to derive a reasonable solution based on some prior assumptions from infinite numbers of possible images that can explain measurements.

In radio interferometry, the most traditional and widely-used algorithms are inverse-modeling approaches collectively called "CLEAN methods” that greedily derive a sparse reconstruction. In the last decade, various scientific demands on image utilization and the limitation of CLEAN methods have driven the development of newer techniques, taking forward-modeling approaches where images are direct fit to data through the measurement equation. The latter includes regularized maximum likelihood methods used to reconstruct the famous first images of a black hole with the Event Horizon Telescope.

This talk will give an overview of various radio interferometric imaging techniques used in modern radio astronomy, from CLEAN methods to newer forward-modeling approaches. Along with each technique, I will introduce what kind of underlying challenges have motivated the development of the newer forward-modeling techniques. In the end, I will discuss various frontiers of imaging techniques developments required by next-generation instruments, as viewed by a radio astronomer.

Video Recording

Brad Sutton
University of Illinois, Urbana-Champaign

Nyquist sampling requirements for imaging have traditionally been thought of on an image-to-image basis where each sampling location of the image is assumed to be independent. This creates significant sampling requirements and has led to techniques like magnetic resonance imaging (MRI) being very slow as it takes time to visit each sample location. More recently there has been an explosion in techniques that leverage correlations in the data itself, or properties of medical images overall, to significantly decrease sampling requirements and enable high speed imaging. In this talk, I will describe the low rank model that we use in several applications of MRI, the partial separability model, and demonstrate how we can separately sample space and time for our high spatiotemporal resolution scans. I will show example applications from brain imaging and dynamic speech imaging.

Video Recording

Maxim Voronkov
CSIRO Astronomy & Space Science

Aperture synthesis is an indirect imaging technique of radio astronomy based on the van Cittert-Zernike theorem, which, under some assumptions, relates the sky brightness distribution and cross-correlations of voltages measured at individual antennas by the Fourier transform. All modern and future radio interferometers (such as LOFAR, ASKAP, SKA, MeerKAT, ALMA, ngVLA) are pushing the boundaries of these assumptions and produce a deluge of raw data to be processed. It is now often the software, which becomes the limitation of scientific capabilities of the instrument. Moreover, the data rate often necessitates distributed processing in a high-performance computing (HPC) environment, which, in some cases, presents new challenges for the algorithm development due to the presence of barriers and parts that are difficult to parallelize. I will review the steps typically involved in making an image from measured radio interferometric data, including advanced gridding techniques (such as w-projection). It will set the scene for discussions of the current issues in the area of algorithm development, related HPC challenges and future directions.

Video Recording

Saiprasad Ravishankar
Michigan State University

Recent machine learning-based reconstruction methods exploit paired (e.g., corresponding regular-dose and low-dose CT scans) or unpaired datasets to learn data models or reconstruction mappings. Incorporating additional priors into the learning framework could enable models learned with limited training data to generalize better.

In this talk, we will first review an approach for model-based image reconstruction (MBIR), where the sparsity-based regularizer's parameters are learned in an unsupervised manner from a small set of high-quality images. We pre-learn a union of sparsifying transforms that can cluster image patches into multiple groups, with a specific transform well-matched to each group. When incorporated in the regularizer in MBIR, the learned transforms provide much better image reconstructions in CT compared to conventional filtered backprojection and non-adaptive regularization methods especially at low X-ray doses.

We then extend the approach to a unified supervised-unsupervised (SUPER) learning-based scheme that combines classical MBIR optimization and unsupervised transform learning regularization together with supervised deep learning in a common formulation. We provide multiple interpretations of the resulting scheme from fixed point iteration analysis or bilevel training optimization and show that with limited training data, it provides much better CT image reconstructions at low X-ray doses than the constituent supervised or unsupervised schemes. Finally, we present a combined supervised-unsupervised framework involving dictionary-based blind or on-the-fly learning and deep supervised learning for MR image reconstruction from under-sampled k-space data. The results show that the image features captured by the blind dictionary-based approach complement deep network learned features to provide significantly better reconstructions with limited training data.

Video Recording

Russ Taylor
UCT-UWC-SKA South Africa

The first decades of this century have seen a tremendous advance in information and digital technologies impacting scientific inquiry The next generation of radio astronomy facilities are harvesting these advances to create data monsters, giving rise to some of the biggest data volumes in science of the coming decades. The challenge presented by the tremendous growth in data generating capacity is exacerbated by the rising research mode of “survey” science, in which large blocks of observing time are devoted to key projects generating vast data sets serving teams of researchers distributed around the globe. At the same time, radio astronomy data must be fused with multi-wavelength data from ground and space observatories to achieve our ambitious science goals.

These combined challenges call for innovation in technologies and approaches to visualization of radio astronomy data. Fortunately the 3^rd and 4^th industrial revolution technologies that generate problem also provide pathways to their solutions. I will discuss the visual analytics challenges and the approaches being developed, including new data structures, cloud platform technologies and virtual reality.

Video Recording

Jeff Mather
Mathworks

The analysis and segmentation of digital pathology images has many challenges. Files are large and often contain imagery stored in a pyramid of multiple resolution levels. The most detailed of these routinely require 30 gigabytes of RAM or more to load fully into main memory and many times more to process naively. The multiple resolution levels do not always cover the same “real world” spatial extents and computing corresponding locations within these image levels is error-prone. Reading from some of these proprietary file formats can be difficult and slow.

Despite these hurdles, there are several tasks that researchers and clinicians routinely need to perform: segmenting objects, identifying features, aggregating details, labeling ground truth, and so on. Some of the most common end goals are assessing tissue health, evaluating drug efficacy, improving microscope design, etc. Many of these tasks are being automated with deep convolutional neural networks.

This talk will present some techniques to overcome the challenges listed above, with an emphasis on speed and throughput. It will also explore how the unique multi-resolution nature of pathology imagery can be used to improve results and reduce processing time. For example, identifying high information content areas in lower resolution views can limit time-intensive processing to only those regions. Additionally, we will show how to chunk up a single high-resolution image to create a large collection of normal sized images to train a semantic segmentation or object detection network… creating tens of thousands of training images even before data augmentation. We will pay special attention to the challenges of class imbalance.

Video Recording

Beatriz Paniagua
Kitware, Inc

The fields of radioastronomy and medical imaging are often limited by a researcher's access to computational tools that: 1) operate in 3 or more dimensions, 2) scale effectively to process the large volumes obtained from rich data sources, 3) provide visualization for remote compute and data resources, and 4) provide an interface that is simple to learn and enables quick iteration during experimental investigations. This lack of analytic tooling to effectively mine the information from high-resolution radioastronomical and bioimaging data acquisition technologies prevents researchers from unlocking quantitative relationships in this data.

This talk will present novel computational tools that address these needs and have the potential to advance the frontiers of understanding across both fields. These 3D visualization and analysis tools, such as Kitware’s Paraview and 3DSlicer, Medical Open Network for Artificial Intelligence (MONAI) or the Insight Toolkit (ITK) and its Jupyter Lab Notebook widgets, called itkwidgets, integrate with computational tools from the scientific Python and machine learning ecosystems and work in remote distributed computing contexts. Preliminary results will be presented on the analysis and characterization of large medical imaging volumes. The talk will conclude with future work and ideas on how to draw connections between the fields of radioastronomy and medical imaging.

Video Recording