Graduate student

The scanning transmission electron microscope (STEM) can reveal the atomic structure of materials at a very high spatial resolution, and the associated spectroscopic methods available allow mapping of composition and chemistry along with optical, electrical, and magnetic properties at the atomic scale. When combined with tomographic techniques, these measurements can be performed in 3D.  There are several roadblocks preventing the more widespread use of STEM techniques for solving real-world problems in an industrial setting.  The instruments are expensive, and the analytical time must be made as short as possible.  Quantification of data can be laborious, lacks repeatability, and the high energy electron beam employed in STEM can permanently alter or even destroy the specimen in the region being analyzed.

Artificial intelligence (AI) may be used to isolate features of interest from instrumental artifacts and remove noise in 3-D tomographic reconstructions from electron microscopy images.  AI may also enable the collection of data with as few electrons impinging on the specimen as possible. An AI system can be integrated with a microscope’s control and data readout to dynamically alter the instrumental parameters based on the characteristics of the output data to maximize information extraction while minimizing electron-specimen interaction. The main limitation of AI approaches to electron microscopy is the lack of training data, especially when high-quality training data is needed. 

A three-pronged approach is envisioned for this project to enable real-time data analysis and efficient image feature extraction methods, leading to less invasive measurements. The successful candidate will design and implement the machine learning (ML) approaches for all three of these strategies.

The first approach involves using conventional iterative techniques to reconstruct the 3D images, followed by image-to-image neural networks to denoise the resulting images and remove spurious artifacts. The accuracy of current 3D reconstruction methods depends on the number of images used for reconstruction and the fraction of the 3D geometry captured by these images. As we limit the amount of data, we expect image reconstruction noise and artifacts to increase.

The second approach uses limited sinogram data as input to image-to-image regression networks that are trained to interpolate the missing data. The idea is to use a full set of images (e.g., images taken at every angle between 0 and 180) and apply iterative techniques to produce a reconstruction of those data, then use a subset of the full sinogram data as input to a neural network that will predict the full set.

The third approach explores the use of AI image-to-image networks that take sinogram data as input and directly output the reconstructed images, thereby minimizing the artifacts produced by iterative reconstruction methods.