Three Dimensional Reconstruction of Single Particle Specimens using Reference Projections

Overview

This page describes methods for creating a 3D reconstruction of a ribosome from electron micrographs. Once a set of particle images has been obtained, an initial 3D reconstruction is calculated using coarse projection angles. This is followed by refinement, which iteratively adjusts the angles for finer resolution. The following procedures are described in greater detail below.

Contrast Transfer Function

(CTF) is estimated for each micrograph, and the micrographs are placed in defocus groups.

Particle Picking and Selection

identifies likely particles from the micrographs, and extracts small images of individual particles.

Alignment

The particle images are aligned to reference projections through shifts and rotations.

Averages

The averages of each reference view are computed, to assess the distribution of projection views.

3D Reconstruction

An initial volume is calculated by back projection, along with an estimate of resolution.

Refinement

Initial alignments are iteratively improved, as each projection is given a chance "to find a better home" in terms of its orientation and phase origin, thus improving the resolution of the initial reconstruction.

Additional methods

Other useful methods, such as classification, difference maps, and frequency correction by X-ray scattering. Additional details about these methods may be found at the techniques page.

A flowchart of operations

For further information:

• See the reconstruction strategies page and the glossary.
• Penczek et al. (1992) describe the technique of 3D reconstruction from particle images.
• Penczek et al. (1994) describe reconstruction using projection matching (the approach outlined below).
• Three-Dimensional Electron Microscopy of Macromolecular Assemblies Oxford University Press, (February 2006)
• More references
• Information about images from the ZI scanner
• Batch files for circular images from Quantifoil grids .

The data below is sometimes shown plotted with gnuplot. An online manual for gnuplot is available. Much more about gnuplot is at Gnuplot Central

In the sections below, various file types are denoted by different fonts:

• Each step in the process is bold and marked by a bullet
  procedure files are links,
  output files created by that procedure file are marked in green.
  Sometimes more than one approach is available. Lists starting with lowercase letters indicate that you should choose
  1. one approach, or
  2. the other approach,
  but not both (unless you want to compare the two techniques!)
  Graphical tools : some results can be analyzed with graphical interfaces, if they have been installed.

• If you are running SPIDER at the Unix prompt, then read below to learn how to execute SPIDER procedure files.
• If you are using SPIRE, then skip this section and read how to run SPIDER procedure files in SPIRE.

Processing steps are carried out by procedure files, which run many SPIDER operations automatically. To use a procedure in this document, click the procedure filename and copy the procedure to your current working directory. At the beginning of each procedure, there is a list of parameters that can be adjusted according to the particular project. Some of the procedures will call a corresponding procedure (listed below the procedure filename). You need not change anything in the procedures. Use the following format to run a SPIDER procedure:

where spi is the procedure file extension, dat is the project data file extension, and proc.spi is the procedure file.

• Create a project directory with the required subdirectories and procedure files

  The procedure files below are packed into the zipped tar file spiproject.tar.gz (for the archive of older procedure files, click here). Download this file to your working directory and unpack it:
       gunzip spiproject.tar.gz; tar xvf spiproject.tar
  It will create a directory called myproject, containing the procedure files and subdirectories for a reconstruction project. Rename myproject to the name of your project directory, usually the same three letters as the data extension, e.g.:
       mv myproject acn

Some information is stored in project-wide document files that are used by many procedure files. The following files should be run in the top-level project directory:

• Create a parameter document file

  Many procedure files read project parameters from this doc file. Run the interactive procedure makeparams.spi in the top directory of your project. It makes a document file:
  

  ¤

  params:

  A doc. file containing the reconstruction parameters

• Create a selection doc file for micrograph file numbers

  The procedure makefilelist.spi creates a micrograph selection doc file in the top-level project directory:
  

  ¤

  sel_micro:

  Micrograph selection doc file listing the numbers of the micrograph files to be processed.

  The text part of the filenames are entered as templates in the top level procedure files - see below). If you need to exclude files from this list, use a text editor to delete the appropriate lines, then use the 'DOC REN' operation in SPIDER to renumber lines in the doc file.

• Files of the digitized micrographs should be in the Micrographs directory. Because they require so much disk space, it may be desirable to keep the Micrographs in another location, and make Micrographs a symbolic link. In that case, cd to your project directory, remove Micrographs, and make a symbolic link:
        rmdir Micrographs
      ln -s /the/actual/location/of/the/micrographs Micrographs
• A reference volume is required. If the dimensions of your reference volume do not match the window size (key 17 in the parameter file), then you must run the procedure resizevol.spi. in the top-level project directory to resize the volume:< br/>

  ¤

  reference_volume:

  Resized reference volume.

After running each procedure file, check the output files to make sure the results are sensible. If you are not sure what is "sensible", ask an expert.

For more information, see the
     contrast transfer function correction tutorial,
     ctfdemo, a program that demonstrates the effects of various parameters on the CTF, and
     Penczek et al. 1997 for a fuller discussion.

• Calculate the power spectrum for each micrograph

  power.spi, and its procedure power_p1.spi, read each micrograph and compute an averaged power spectrum for it. Each micrograph has two corresponding output files:

  ¤	power/pw_avg***	2D power spectra, each is an average of many smaller power spectra calculated over the surface of the micrograph.
  ¤	power/roo***	Doc files of 1D profiles of the rotationally averaged power spectra. These doc files can be plotted with gnuplot or pyplot

• Check the rings in the power spectrum images.

  Make a montage using WEB (reduction factor 4) to look at these power spectra. They are equivalent to diffraction patterns obtained in the optical diffractometer. It is absolutely important that you screen these spectra before proceeding any further. Things to look out for:

  • Are the Thon rings round?
  • Do they extend to high resolution?
  
  You can discard micrographs that show one of the following artifacts:
  • Rings that are cut off unidirectionally(evidence of drift)
  • Rings that are elliptical, or even hyperbolic (evidence of astigmatism).
  See this example:

  

• Determine defocus

  Use one of the following methods to estimate defocus for each micrograph:

  1. defocus.spi estimates defocus with SPIDER operation TF ED, creating:
     

     ¤	defocus:	A doc file of defocus and astigmatism values.
     ¤	power/ctf***:	Doc files of the background-subtracted 1D power spectra.

     It uses the procedure: calcparms.spi, which checks values in the parameter file.
     The ctf*** files can be analyzed with ctfmatch.py
  2. CTF from doc file in WEB. Manually fitting a CTF model to the 1D profiles.
  3. mrc.spi a procedure file that calls the MRC program ctffind2. It estimates both the defocus and astigmatism, and creates the output file: defocus. It calls ctffind, a shell script, and readmrc.py, a python script. See the defmrc manual page for more information.

• Assign defocus groups according to the power spectrum.

  defsort.spi: reads the "defocus" doc file generated by one of the above methods, and creates:
  

  ¤

  def_sort:

  A doc file with the same information sorted by defocus.

  It also tentatively assigns each micrograph to a defocus group. These automated defocus group assignments are only a starting point, however. You must manually inspect the defocus groupings to see which micrographs belong together, ensuring that they are "in phase".

  Spectral profiles (that is, the roo*** or ctf*** files) from the same defocus group should be plotted together, to ensure that their zeroes (minima) line up, and do not cancel out. Plot the power spectrum for each micrograph and assign them to defocus groups such that the curves are in phase. Use either:

  1. ctfgroup.py
  2. pyplot
  3. gnuplot
     Gnuplot commands can be saved in a text file, such as plotpower, then run in gnuplot using the load command:
     gnuplot> load 'plotpower'

     

  When you have determined a satisfactory set of groupings, update the docfile def_sort, which is used in the next step.

• Compute averages for each defocus group

  When you have finalized the values in def_sort.ext, compute the averages for each defocus group using defavg.spi which creates:
  

  ¤	def_avg:	A selection doc file of defocus group averages.
  ¤	order_defgrps:	A selection doc file of defocus group averages.
  ¤	dat_group:	A doc file of defocus group averages.
  ¤	dat_mic:	A doc file of micrograph defocus values.

• Select a background noise file

  A noise image is required to normalize the backgrounds of the particle images. Do one of the following:
  1. Use a noise image from a previous project (this is the easier way). The window sizes do not need to match.
  2. noise.spi creates:
     

     ¤

     noise.ext:

     A noise image.

     This is selected at random from a set of regions in the micrograph that appear not to contain any particles (the files tmpnoise/noi***). View the selected noise file; if it has any structure (particle, junk), then select one of the other candidates and copy it to noise.ext. This procedure file requires a micrograph in SPIDER format. If your micrographs are tif and/or gzipped files, use convert.spi and its procedure convert_p.spi to convert one of these images to SPIDER format.

• Run automatic phase of particle selection.

  Either of the following two approaches can be used. They both require the converting procedure convert_p.spi
  1. pick.spi and its procedure pick_p.spi, Apply correlation with a Gaussian blob.
     The micrographs are decimated by 1/4, Fourier filtered with a Gaussian low pass filter, and searched for peaks over regions corresponding to particle dimensions. Particles are windowed out and centered, then peak locations are corrected, and particles are windowed again. This procedure uses histogram equalization. Output files:
     

     ¤	winser_****:	Stacked particle images.
     ¤	coords/ndc***:	Peak coordinates from 1/4 decimated image.
     ¤	coords/sndc***:	Top left pixel coordinates for particle images.

  2. lfc_pick.spi (and procedure pickparticle.spi) use the Local Fast Correlation algorithm. This procedure uses local cross-correlation calculated using Fourier methods outlined by Roseman (2003). This implementation is described in Rath and Frank (2004). A reference volume is required to generate projections at desired Eulerian angles for searching different orientations of the search image in the micrograph. Output files:
     

     ¤	win/winser_***:	Stacks of particle images for each micrograph.
     ¤	win/sel_particle_***:	Particle selection doc file for each micrograph.
     ¤	coords/sndc***:	Pixel coordinates for center of particle images.

• [Optional] Low-pass filter particle images before particle selection

  1. The procedure pfilt.spi filters a set of images, adjusting filter settings for each micrograph, based on its defocus.
  2. The procedure pfilt2.spi is a simpler version.
  Both procedure files create:
  

  ¤

  flt/winser_***:

  Stacks of low-pass filtered particle images.

  Manual selection of good particles (the next step) is easier if the particles have been low-pass filtered beforehand. Use the filtered particles, not the raw particles, during selection in WEB.

• Verify the automatically selected particles

  Sort through the output files to eliminate any non-particles. This can be done in WEB using the Categorize from sequential montage operation. When the particle image files are displayed on the screen, use the mouse to click on each good particle. For each micrograph, save your selections to a file called good/good***. (More details in the SPIDER FAQ and partpick.html). This procedure creates:
  

  ¤

  good/good***:

  A set of particle selection doc files, one for each micrograph.

• Remove any duplicated particles and list number of picked and selected particles by micrograph

  The procedure: renumber.spi creates:
  

  ¤	good/ngood***:	A set of particle selection doc files, one for each micrograph, listing unduplicated selected particle numbers.
  ¤	percent_selected:	A doc file listing percentages of automatically picked vs manually selected particles for each micrograph and overall.

  When manually selecting particles in WEB, sometimes a particle is accidently double clicked. This procedure will delete the appropriate lines from the file. Run this procedure, even you you don't think you made any mistakes - subsequent procedure files will require the ngood*** files as input.

Reference projections
       Reference projections (images) are views of the object at known angles. In what follows, angular accuracy is set to 15 degrees, which results in 83 reference projections. A reference volume is needed in the top level project directory to create the projections. The row and column dimensions of the reference volume must match the dimensions of the particle windows. (see resizevol.spi)

• Create reference projections for alignment from a reference volume

  refproj.spi creates two sets of files:
  

  ¤	refangles:	A doc. file listing the three Euler angles for each of the projections, generated with the SPIDER operation VO EA
  ¤	projs/prj_****:	Stacked reference images produced with PJ 3Q using the above file.

  There are two types of reference projections:
  1. You can use the same stack of references for all defocus groups, by setting register [usectf] = 0. They will be in the file projs/prj_001
  2. Generate more realistic references by multiplying them by the CTF (the default). There will be a stack of reference projections for each defocus group.

• Create lists of images in each defocus group

  sel_by_group.spi creates selection document files and a group summary file:
  

  ¤	sel_particles_***:	A doc file for each defocus group, listing all the particles in that group.
  ¤	sel_group:	A summary doc file with 4 columns listing: Defocus group, Number of particles, Cumulative number of particles, and Average defocus value for each group.

• Create stacks of images for each defocus group

  win2stk.spi creates stacks:
  

  ¤

  data***:

  A stack file for each defocus group, containing all particle images from that group.

• Align particles to the reference projections

  1. If you are using the same reference stack for each defocus group:
     apsh.spi uses AP SH, to create an alignment parameters document file and a stack of the aligned images.
     

     ¤	align_01_001:	Shift and rotation parameters for the best matched projection.
     ¤	dala01_001:	Aligned stack of the particle images.

  2. If you have references for each defocus group:
     apshgrp.spi uses AP SH, to create a set of alignment parameter document files and stacks of aligned images.
     

     ¤	align_01_***:	Shift and rotation parameters for the best matched projections.
     ¤	dala01_***:	Aligned stacks of the particle images.

     If you are using Publish and Subscribe on a Cluster, activate PubSub within apshgrp.spi procedure heading.

• Make particle selection document files listing particles for each projection

  select.spi creates the following doc files:
  

  ¤	df***/how_many:	Lists number of particles associated with each reference view for each group.
  ¤	df***/select/sel???:	Lists the particles associated with each reference view for each group.
  ¤	how_many:	Summary file listing total number of particles associated with each reference view.

• Create average images for all projection groups and combine all alignment parameter files into a single summary file (if necessary).

  average.spi creates the following files for a selected defocus group.

  ¤	avg/avg***:	Average images for each projection group.
  ¤	avg/var***:	Variance images for each projection group.
  ¤	align_01_all:	Merged alignment parameters from all defocus groups. (Not necessary if only a single group is used).

  Use the Montage operation in WEB to view the set of average images. Projection views with many particles will have averages that reasemble the reference projection. Projection views with few particles will be noisy.

The next several operations help to identify a threshold correlation coefficient that describes true particles as opposed to any erroneously selected particles. Create histograms of each defocus group that plots the number of particles vs. the cross correlation value. For example, if the histograms display a bimodal distribution, the higher peak may correspond to actual particles, while the lower peak may show noise. The threshold should be chosen between two such peaks.

• Identify a threshold correlation coefficient for selecting true particles

  cchistogram.spi creates histograms and a gnuplot control script:

  ¤	hist/cchist_***:	Set of histograms by defocus group.
  ¤	hist/cchist_all:	Overall histogram.
  ¤	plot_hist:	Gnuplot script for displaying overall histogram.

  To view the overall histgram plot issue following system command: gnuplot -persist plot_hist.ext >
  

View the particles corresponding to each reference view.
Use the Montage from doc file operation in WEB to display the aligned particles identified in the selection doc file (sel001.ext for example). The greater the correlation coefficient value, the more similiar the particle will look to the reference projection. Identify a minimum correlation coefficient that describes true particles as opposed to erroneously selected particles. Select Compute Average to view an average of the particles displayed.

• Compute correlation thresholds for discarding particles

  ccthresh.spi, given a percent cutoff level, creates threshold and select doc file:
  

  ¤	thresh	Thresholds for each defocus group.
  ¤	df***/seltotal	Particle selection files.

  E.g., if a threshold of 0.20 is selected, then 20% of all particles with low correlation values will be rejected. The doc file lists, for each defocus group, the cutoff correlation threshold, and the numbers of particles above and below the threshold. NB, BECAUSE THE HISTOGRAM 'BINS' THE VALUES, THE NUMBERS OF PARTICLES ARE NOT EXACT, AND THEREFORE WILL NOT EXACTLY MATCH THE OUTPUT OF dftotals.spi.

  The correlation thresholds in thresh may be edited, if you wish to use different cutoff levels.

• Select particles above the correlation thresholds

  dftotals.spi applies the cutoff levels from thresh to create:
  

  ¤

  sel_particles_***

  Particle selection files.

  The selection files list particles with correlation values greater than the threshold and are used to select particles for inclusion in the reconstruction.

• Check the distribution of reference views (angular directions)

  plotview.spi creates a text file of gnuplot commands:
  

  ¤

  plot_view

  gnuplot commands

  which plots the file how_many.ext, showing the number of particles vs projection view.

  display.spi and its procedure display_p.spi create:
  

  ¤

  display/cndis***:

  SPIDER image files showing distribution of sample images among various reference projections.

  These files show the various angular reference groups represented by small circles. The radii of these circles are proportional to the number of particles in each group.

  Use the Montage operation in WEB to display the plots.

• [Optional] Limit strongly over-represented angular projections

  bestim.spi creates:
  

  ¤	sel_group_cclim	Group selection file for specified defocus group
  ¤	sel_particles_***	Particle selection file for specified defocus group

  The selection limits the numbers of images per reference view direction to a certain specified count. This will limit strongly over-represented angular projections which may cause a shape error in the reconstruction. Continue with the set of limited particles.

• Split the particle images, create paired reconstructions, and compute resolution for each defocus group

  deffsc.spi creates two selection doc files for particles by randomly partitioning the images into two equal sets:

  ¤	df***/seleven:	Odd numbered images for each defocus group.
  ¤	df***/selodd:	Even numbered images for each defocus group.

  It then creates two half-reconstructions for each defocus group using back projection:
  

  ¤	df***/vol001_odd:	Reconstructed volume from odd images for each defocus group.
  ¤	df***/vol001_even:	Reconstructed volume from even images for each defocus group.

  Finally it computes the resolution of each defocus group by comparing the two half volumes and creates:
  

  ¤

  df***/doccmp001:

  Doc files containing FSC curves for each defocus group:

• Check the defocus group volumes by viewing a central slice from each volume

  slices.spi extracts a central slice from a volume in each defocus group:
  

  ¤

  slices/slice***:

  Central slice from each defocus group volume.

  Use the Montage operation in WEB to view the slices. Ideally, each image should show a particle isolated from its surroundings, exhibiting some internal structure - somewhat like the reference images. If any images are blank or otherwise unusual, go back and check the processing of that defocus group.

• Apply CTF correction, construct an initial volume, and compute the combined resolution

  The procedure ctf.spi applies CTF correction to all the defocus group odd/even volumes (see the TF CTS operation), adds together the odd and even volumes to make a volume for the entire data set. Finally it calculates the FSC resolution curve for the combined data set (see RF 3 operation) creating the following files:
  

  ¤	combires:	Combined resolution doc file.
  ¤	vol001:	3D structure from entire set of particles.

  This structure is known as the initial volume and will be used as the input for the Refinement step.

• Determine resolution of the initial volume

  View the combined resolution doc file to determine where the Fourier shell correlation drops below threshold (0.5), and identify the corresponding spatial frequency. Then, resolution = pixel size/spatial frequency.
  res.spi creates the doc file:
  

  ¤

  resolution:

  Doc. file ontaining the cutoff, the corresponding normalized frequency, and resolution (in Angstroms).

  plotres.spi creates a text file of gnuplot commands:
  

  ¤

  plot_res:

  Gnuplot script for plotting the resolution curve of each defocus group, along with the combined resolution curve.

  

• Filter the initial volume

  Information at spatial frequencies above the cutoff value are considered to be uncorrelated noise, and must be filtered out.
  filt.spi filters the initial volume at the cutoff frequency.

  ¤

  volfq001:

  Filtered volume.

  Use the Surface operation in WEB to view the filtered volume.

• A more detailed set of instructions for refinement is given on the Refinement page.

• Collect necessary files in the Refinement/input directory

  copyin.pam assembles the necessary input files:

  ¤	input:	Directory for input files.
  ¤	input/order_select:	Group selection file.
  ¤	input/order_select_sort:	Sorted group selection file.
  ¤	input/vol01:	Initial starting volume.
  ¤	input/select_{***[grp]}:	Particle selection files.
  ¤	input/align_01_{***[grp]}:	Alignment parameter files.
  ¤	input/data{***[grp]}:	Unaligned stacked image files.
  ¤	input/dala01_{***[grp]}:	Initial aligned stacked image files.

• Set the parameters in refine_settings.pam.

• Run refine.pam (or pub_refine.pam if running on the cluster with PubSub).

  There are a number of sub-procedures used in refinement: refine_settings.pam, prepare.pam, grploop.pam, mergegroups.pam, saveresp.pam, endmerge.pam, and endrefine.pam
  If you are running on a cluster with PubSub you will also need: publish.perl, pub_refine_start.pam, pub_refine_doc_sync.pam, and pub_refine_wait.pam.
  Further Details are given in the Refinement page.
  Among the outputs of refinement are image and document files with '***' denoting group number and '##' denoting iteration number:
  

  ¤	final/align_##_***:	Alignment parameter doc files
  ¤	final/dres##_***:	Resolution curve doc. file
  ¤	final/val##:	Unfiltered volume
  ¤	final/vol##:	Filtered volume from even numbered images
  ¤	final/vol##_odd:	Filtered volume from odd numbered images
  ¤	final/vol##_even:	Filtered volume from all images
  ¤	final/defgrp***/dbpr##:	Resolution curve doc file
  ¤	final/defgrp***/bpr##:	Group volume
  ¤	final/defgrp***/bpr_##_odd:	Group volume from odd numbered images
  ¤	final/defgrp***/bpr_##_even:	Group volume from even numbered images
  ¤	final/bpr##:	Final filtered volume from all images
  ¤	final/bpr1##:	Final filtered volume from even numbered images
  ¤	final/bpr2##:	Final filtered volume from odd numbered images
  ¤	final/dbpr##:	Overall resolution curve doc file
  ¤	resolutions:	Doc file containing resolution at each iteration.

• Plot the refinement resolution curves

  The procedure plotref.spi creates two text files of gnuplot commands:
  

  ¤	plot_refi:	Plots combined resolution curve for each iteration of refinement.
  ¤	plot_refd:	Plots resolution curve for each defocus group for the last iteration.

• [Optional] View angular data output from Refinement procedure files

  The procedure angdisp.spi creates angular distribution images:
  

  ¤

  disp_ii_***:

  Set of images showing angular positions from the align_ii_*** refined alignment parameter output files.

  The procedure angdiff.spi creates:

  ¤

  diff_ii_***:

  Image showing changes in angular positions from iteration n-1 to iteration n.

---

Other Useful Methods

Create tar files for backup

backup a shell script, puts a number of vital program files into tar files, which can be backed up on tape. Save the shell script, make it executable:

chmod 775 backup

and run it in the project directory. It creates 2 tar files order.tar, and project.tar

Difference Maps

Differences between volumes can be assessed by subtracting one volume from the other.

Frequency amplitude correction with X-ray scattering data

Enhance the Fourier amplitudes of a reconstructed cryo-EM volume so they more closely resemble those of experimental low-angle X-ray scattering data.

Classification of Particles

This method can be used to investigate differences between particles.

See more approaches at the techniques page

---

References

1. Penczek, P., M. Rademacher, and J. Frank (1992) Three-dimensional reconstruction of single particles embedded in ice. Ultramicroscopy 40:33-53.
2. Penczek, P., Grassucci, R.A. and Frank, J. (1994) The ribosome at improved resolution: New techniques for merging and orientation refinement in 3D cryo-electron microscopy of biological particles Ultramicroscopy 53, 251-270.
3. P.A. Penczek, J. Zhu, R. Schröder, J. Frank (1997) Three Dimensional Reconstruction with Contrast Transfer Compensation from Defocus Series Special Issue on Signal and Image Processing, Scanning Microscopy Volume 11, 1997, page 147.
4. B.K. Rath and J. Frank (2004) Journal of Structural Biology, Volume 145, Issues 1-2, January 2004, Pages 84-90.
5. Alan Roseman (2003) Ultramicroscopy, Vol 94 (3-4), pp. 225-236

Source: mr.html     Page updated: 04/05/07    

© Copyright notice       Enquiries: spider@wadsworth.org