Python scripting and Spire compatibility

Back

Spider/Spiderscripts.py

Functions in Spiderscripts
Parameter file utilities

Spire, the SPIDER Reconstruction Engine, requires that batch files have a specially formatted header indicating the input and output filenames, as well as numerical parameters. This header format allows Spire to present the batch file in a graphical user interface so users can conveniently change inputs and filenames. The details of the header format are in the Spire documentation, but suffice it to say, they easily map into Python with simple adjustments to the syntax.

Python scripts for processing document files should import the main SPIDER Python Library:
from Spider.Spiderutils import *
and they should import some accessory functions for handling batch files (see coding examples):
from Spider.Spiderscripts import *

An example Python script header:
# ----------- Input files --------------

FILENUMS = "../filenums"    # file numbers

coords = "coords/sndc****"  # coordinate docfiles with particle numbers

# ----------- Output files --------------

outfile = "order_picked"    # output doc file

# -------------- END BATCH HEADER -------------

The file extension must not be present, since it changes with each project. Analogous to the execution of a SPIDER batch file, the extension is passed into the script on the command line, e.g., myscript.py dat
The utility function extFromCommand fetches the data extension for use in the script (see example scripts).

The file numbering document file, FILENUMS, has a special meaning to Spire, which is retained in Python scripts.

Finally, there is a class available for the parameter document file, which simplifies access to items in the parameters document file. The line
P = Parameters("../params.dat")
creates a parameter object, P, whichs allows one to refer to items using the dot notation, e.g., P.pixelsize, P.magnification, etc.


Functions in Spiderscripts


The functions below are available with the following import statement:
from Spider.Spiderscripts import *
extFromCommand (sys.argv)
Gets the data extension, either from the command line or from a user prompt.
The argument must always be sys.argv, the array of command line arguments.
The data extension must be three letters and it always starts with a dot.
# -------------- END BATCH HEADER -------------
ext = extFromCommand(sys.argv)

# then you have to add the extension to each filename (or use putExtension)
FILENUMS = FILENUMS + ext
outfile += ext  # shorthand for "outfile = outfile + ext"
See the example defsort.py

putExtension (ext, force=0)
Adds the data extension to filename strings in the batch file header immediately above it.
Only works on strings, it won't add extension to a number (e.g., x11 = 77.0).
It ignores filenames that already have an extension, such as 'plotview.gp' (unless force=1).
Ignores filenames if the comment contains the word 'directory' (or 'directories').
Ignores all filenames in a section heading containing the word 'directory' (or 'directories').
# -------------- END BATCH HEADER -------------
ext = extFromCommand(sys.argv)

putExtension(ext)
# now you don't need to do the following:
#FILENUMS += ext
#outfile += ext
See the example defavg.py

useFilenums (filenums, template)
Returns a list of filenames, with file numbers substituted into template asterisks.
filenums: list of filenumbers (or it could be the filenums doc file).
template: a SPIDER filename template w/ asteriskse.g., img****.dat
>>> useFilenums([1,3,44], "img***.dat")
['img001.dat', 'img003.dat', 'img044.dat']
See the example pnums.py


Parameter file utilities

Two accessories are provided for reading parameter document files:
1) the function getParameters returns a dictionary of values and associated comments. This is the more general solution.
2) The class Parameters is specialized for SPIDER single particle reconstruction parameters.
They work with the standard doc file comment lines or with the newer Spire one-liner comment lines.

getParameters (parmfile)
Reads a parameter document file, returns parameters in a Python dictionary.
The parameter dictionary is indexed by the doc file keys (integers).
Each dictionary element is a list of the form [key, value, comment], where key is an integer, value is a floating point number, and comment is a string.
>>> d = getParameters("params.dat")
>>> d[6]
[6, 200.0, 'electron energy (kV)']
>>> keys = d.keys()
>>> keys.sort()
>>> for key in keys:
...    print d[key]
...
[1, 0.0, 'zip flag']
[2, 0.0, 'file format']
[3, 0.0, 'width (pixels)']
[4, 0.0, 'height (pixels)']
[5, 4.78, 'pixel size (A)']
[6, 200.0, 'electron energy (kV)']
[7, 2.0, 'spherical aberration (mm)']
[8, 0.0, 'source size (1/A)']
[9, 0.0, 'defocus spread (A)']
[10, 0.0, 'astigmatism (A)']
[11, 0.0, 'azimuth (degrees)']
[12, 0.1, 'amplitude contrast ratio (0..1)']
[13, 10000.0, 'Gaussian envelope halfwidth (1/A)']
[14, 0.025078, 'lambda']
[15, 0.104603, 'max. spatial frequency']
[16, 0.0, 'reserved']
[17, 75.0, 'window size (pixels)']
[18, 52.0, 'particle size (pixels)']
[19, 0.0, 'magnification']

Parameters (parmfile [, usecomments=0])
A class that reads a parameter doc file, and returns a Parameter object. Attributes of the Parameter object are listed by Object.attribs. The list contains a set of strings which can be directly accessed using the dot notation.
>>> P = Parameters('params.dat')
>>> P.kv
200.0
>>> P.Cs
2.0
>>> P.attribs
['zip', 'scan_format', 'width', 'height', 'pixel_size',
 'kv', 'Cs', 'source_size', 'defocus_spread', 'astigmatism', 'azimuth',
 'acr', 'Gaussian_env', 'wavelength', 'max_spat_freq', 'decimation',
 'window_size', 'particle_size', 'magnification']
Note: by default, this class is specific to the SPIDER single particle reconstruction parameters.
You can use the Parameters class for a different set of parameters, but it requires an attribute name for each parameter. The comments in the parameter file may be used as attributes by setting the keyword usecomments=1. Only the first word in the comment section is used as the attribute. So for the following parameter file:
    1 1    0.000000     ; file format ['SPIDER', 'HiScan', 'ZI scanner']
    2 1    4.780000     ; pixel size (A)
    3 1  200.000000     ; electron energy (kV)
    4 1    0.100000     ; amplitude contrast ratio (0..1)
    5 1    0.025078     ; lambda
    6 1    0.104603     ; max.spat.freq
You could try to use the comments as attributes:
>>> P = Parameters('params.dat', usecomments=1)
RESERVED WORD WARNING: using 'wavelength' instead of 'lambda'
>>> P.attribs
['file', 'pixel', 'electron', 'amplitude', 'wavelength', 'max_spat_freq']
>>> P.wavelength
0.025078
Note that:
- 'wavelength' is substituted for lambda, which is a reserved word in Python.
- Dots are replaced by underscores.
- You might not get something intelligible.