Virtual Expert in the Electron Microscope

Student Project: Virtual Expert in the Electron Microscope



The 21st century is one of the most productive era in the history of drug discovery. It is largely due to the revolutionized field of structural biology, within which we see the molecule structure, we explore its function, and we design macro molecules to cure diseases.

Among all of those structural biology techniques, Single Particle Analysis (SPA) on Transmission Electron Microscope (TEM) is no doubt the leading one. It illuminates a frozen sample using a low-dose electron beam, acquires thousands of 2-dimensional images of the sample, and deduce the 3D structure by advanced image and volume analysis.

This technique is awarded the Noble prize in 2017 for resolving high resolution molecule structures which previously were thought to be impossible.


Left: Typical SPA workflow. Courtesy of Prof. Z. Zhou. Right: The 3D structure of Gamma Secretase, which helped us understand the Alzheimer disease. (A) The SPA structure (B) The atomic structure. Courtesy of Prof. Y. Shi.


Problem Statement

Despite its great achievements, SPA nevertheless has its short-comings: it requires a massive amount of expertise to ensure in every step of data acquisition, only the highest quality information is selected.

A typical decision cycle for a SPA scientist:

– Is the microscope condition good enough?

– If yes, which piece of the prepared samples should I work on first?

– Once decided, which position on this piece of sample should I shine the beam on?

– Once the image is taken, where is the molecule of interest on this image? Is it good enough?

All of those questions can be answered by a well-trained expert but is very difficult to solve using classical algorithms.

A typical decision flow of SPA. From a to e, the scientist selects finer and finer area, and eventually results in the good quality image. Courtesy of Prof. Z. Zhou.

Therefore, to increase the ease of use and assist faster drug design, our ultimate goal is to learn from those experts, and eventually train a virtual expert in the microscope with deep learning technique.


Student project

This project is the first step towards this goal. It includes the following activities for the student:

– Literature study of deep learning techniques and the SPA workflow. Identify the potential candidates for training the virtual scientist.

– Train a neural network for one of the above decision steps.

– Analyze its performance compared to the fully manual approach.

– Write a final report.

Student profile

– Affinity with deep learning / math / image processing / signal processing

– Able to program in one or more languages ( Python / C++ / Matlab )

– Creative, enthusiastic, communicative


About Thermo Fisher Scientific

Thermo Fisher Scientific is the world leader in serving science, with revenues of more than $20 billion and approximately 70,000 employees globally. Our mission is to enable our customers to make the world healthier, cleaner and safer. We help our customers accelerate life sciences research, solve complex analytical challenges, improve patient diagnostics, deliver medicines to market and increase laboratory productivity. Through our premier brands – Thermo Scientific, Applied Biosystems, Invitrogen, Fisher Scientific and Unity Lab Services – we offer an unmatched combination of innovative technologies, purchasing convenience and comprehensive services.

Thermo Fisher Scientific Contact

More information on possibility for projects can be obtained from:

Dr. Ir. Erik Franken

Email: erik.franken <at>

Dr. Yuchen Deng

Email: yuchen.deng <at>