get another manpage
NAME
X-GEN - xgenproc.py
SYNOPSIS
xgenproc [--help] [--version] [-dknsuv] [-cfhijlmpwxyz<val>] [datadir]
DESCRIPTION
"xgenproc.py" is a Python script that allows you to process a single
run of diffraction data through X-GEN, beginning with raw images and
culminating with the production of several ascii files containing
merged, scaled intensity measuremnts. "xgenproc.py" has been tested
primarily on data obtained from the two beamlines of Southeast Regional
Collaborative Access Team, but it should work with data collected on
home sources and from other beamlines too.
The `xgenproc.py' script is found in the executables directory within
which the rest of the X-GEN executables live. On most machines that
directory will either be /usr/bin itself, or it will be /opt/xgen/exec,
or /home/[INSTALLER]/xgen/exec, where [INSTALLER] is the username of
the person who installed X-GEN on your system. The executables,
libraries, and resource files for X-GEN, along with the script itself,
are available for download from http://xgen.iit.edu. Of course, to run
this, you will also need Python. This script has been tested with
Python 2.3.5, and it probably runs on later versions; it may be compat-
ible with earlier versions too.
If you have collected data into a known directory, you should be able
to use this script in a simple manner. First, set your working direc-
tory to the directory where the data are, e.g.
cd /u1/ID7/03_18_05_uXX/Fred/uglyxtal
Then if the data are from the SER-CAT ID line and were collected in
omega, you can process the data by typing "xgenproc.py -s". The script
will work properly with data from most other sites (including the SER-
CAT ID collected in phi) if you simply type "xgenproc.py". The various
run-time parameters will enable you to tailor your run of xgenproc.py
to the specific circumstances under which you're running it.
Frequently asked questions:
+ How do I know if it has worked?
The processing results will be written to a subdirectory of your cur-
rent directory called xgen. If that directory already exists (because
you or someone else has already processed these data before, or has
processed a different dataset in the same directory), then they'll be
written to the directory xgena. If xgena exists, they'll be written to
xgenb, and so on up through xgenz.
So if the output to your screen looks reasonable (the Rmerge values
that you see near the bottom are good), then the processing probably
worked. If you cd to the processing directory (xgen, or xgena, or what-
ever), then if you look at the logfile, you can see how effective the
processing was. The logfile has a name that ends in .xlg.:
% cd xgena
% vi *.xlg
Examine that .xlg file to see the quality of the data. The scaling
results will be near the end.
+ What if that doesn't work?
There are several possible reasons it might not work.
* The software may have failed to find the detector images in the
current directory. If that happens, you may want to specify your
image prefixes explicitly. Thus if your data in
/u1/ID7/03-18_05/UXX/Fred/uglyxtal/ have filenames ugly01.0001,
ugly01.0002, . . . , up through ugly01.0137, then you might want
to try
% xgenproc.py -s -pugly01
If it's still unable to find the images, you may need additional
help from the author (howard@iit.edu).
* It may fail during auto-indexing. There are some special cases
for this problem:</li>
- It may have gotten the stepsize wrong.
Note the rules I've given above. If you collect on the bending-
magnet beamline, or if you collect on the ID line using phi as
your data collection motor, then you don't need to flip the sign
of the stepsize. If you collect on the ID line using omega as
your data collection motor, then you need to specify the sign-
flip:
% xgenproc.py -s
- It may not have found enough reasonable centroids to refine
on.
You can circumvent this by specifying a larger minimum number of
centroids. By default the program searches for spots over seven
images or until it finds 400 centroids, whichever comes last.
If 400 isn't enough, specify a larger number, e.g.
% xgenproc.py -c700
on the BM line or on the ID line with phi data, or
% xgenproc.py -s -c700
on the ID line with omega data.
- The crystal may be twinned or pathological in some other way.
International Tables number. Thus if your crystal is in space-
group P6(2)22 (International Tables # 180), you can specify it
at runtime as:
% xgenproc -i180
You can combine this flag with the others listed aobve if appro-
priate.
* It may integrate data satisfactorily at first, but fail farther
along, with rapidly increasing RMS errors in X, Y, or omega in
the on-the-fly refinements. This sometimes indicates that the
crystal has moved discontinuously somewhere along the line in
data collection. If the failure is more gradual, it may be a
sign of an increase in mosaicity or a problem with icing. You
may be obliged to cut off the integration somewhere before the
end. You can do this by specifying an upper limit to the number
of images to be integrated. Thus if you collected 180 images but
only the first 130 are well-behaved, you can specify that as
% xgenproc -h130
Again, you can combine this flag with others.
* It may integrate satisfactorily, but produce unsatisfactory
R(merge) values in scaling. There are multiple possible causes
for this too:
- The beam-center values are wrong. Look at your .xlg file just
before the last integration; the output from the refinement seg-
ment will contain something that looks like "Start omega: 1.00
chi: 0.00 phi: 0.00 Mainbeam: 1031.0 1041.9" Note that the
Mainbeam position _in pixels_ is specified here. If the Mainbeam
values in this window are seriously off from where we expect
them to be, then we may need to specify them specifically. Here
if we know they really should be [X,Y] = [1021,1048], we can
specify those values at run-time with
% xgenproc -x1021 -y1048
- The spacegroup is wrong. Typically this happens if we have
been too aggressive in assigning a high symmetry when a lower
one is real. Figure out the correct one and specify it:
% xgenproc -i75
- There are icerings in the data that are polluting the results.
This can be readily dealt with in the specific applications
within X-GEN, but not in the one-line processing system.
- The data have been scaled at higher resolution than is really
legitimate. X-GEN by default analyzes the output of the inte-
gration step and estimates the plane-spacing at which the aver-
age value of I/sigma falls to 1.3. It sets the resolution limit
there. That value may be at higher resolution than you really
believe is appropriate. My contention is that you can always
throw out some of your data if there's a little too much
present; you can't get it back if you didn't process at a suffi-
ciently high resolution. If you want to see what the statistics
are like at a different resolution cutoff, you can easily get
those. To do that, change directories to the directory where the
processing was being done, establish the X-GEN environment vari-
ables for your project, and run the X-GEN "stats" application,
specifying the resolution limit that you believe is really
appropriate:
% cd xgena # cd to processing directory
% source *.com # establish environment variables
% stats -on 3.1 # Friedel-mates merged to 3.1A
This tells the application to calculate intensity statistics for
the data out to 3.1 Angstroms, separating Friedel mates for the
R(merge) calculations.
+ Does "xgenproc" have run-time options?
Yes. They're listed under "Options".
+ But I want to merge multiple runs!
This is readily possible with X-GEN, but not through the "xgen-
proc" interface. I'll develop a simple interface that can han-
dle multiple data runs shortly.
+ I want to see the images!
You can readily do so. Change directory to your processing
directory, establish your environment variables, and examine the
images with "pdisplay":
% cd xgen # cd to processing directory
% source *.com # establish environment variables
% pdisplay # examine your images
"pdisplay" has many options that you can explore.
+ I'm stuck in ways that aren't discussed here. What do I do?
Read the full documentation for X-GEN, or call or e-mail Andy:
773-368-5067 (cell), 312-567-5881 (IIT), or howard@iit.edu.
OPTIONS
--help Print this message, and exit.
--version
Print a header indicating the version of X-GEN that is running.
-d Compress the images using the "gzip" algorithm after integra-
tion.
-s Flip the sign of the rotation stepsize from the value that the
program intuits. We try to stay in synch with the stepsize con-
ventions for every goniostat, but if we get it wrong, this
allows recovery.
-u Same as "--help", i.e. print this message and exit.
-v Same as "--version", i.e. print the X-GEN version information
and exit.
-c<val>
Specify the number of centroids required for refinement. In
general the `spots' functionality will search for 300 spots; if
it doesn't find that many in the first seven frames, it will
continue looking until it finds 300. Specifying -c<val> will
direct "xgenproc" to look for a different total.
-f<val>
Specify the name of a detector image to be used as an example of
the names of the remaining images. Thus if the image
george1a.0037 appears in the run, and we want to process the
data from all the images with names of the form george1a.####,
then specifying -fgeorge1a.0037 will accomplish this.
-h<val>
Specify the last image we will process. This enables two differ-
ent capabilities: First, if we are collecting while we're pro-
cessing, we can use this flag to specify the last frame we actu-
ally plan to collect, even if at the time we start running X-
GEN, that frame hasn't been collected yet. Second, if we know
that images 57-180 are garbage, we can specify -h56 to indicate
that we only want to process up through image 56.
-i<val>
Specify an International Tables spacegroup number. If auto-
indexing succeeds, and the Laue group associated with the speci-
fied spacegroup number is a plausible solution to the Bravais-
lattice calculation, then we will impose this spacegroup number
during processing. To specify a rhombohedral indexing of a rhom-
bohedral unit cell, add 100 to the spacegroup; thus R32 indexed
rhombohedrally is -i255.
-j<val>
Remap the detector's geometrical irregularites at each refine-
ment during integration. On the first one, write the results out
to disk; otherwise, simply record them in memory for use within
the application. <val> in this case is the RMS error in (X,Y)
above which the remapping will be performed, in ten-thousandths
of pixels; thus -j3000 corresponds to remapping only if the RMS
error is worse than 0.3 pixels. The default value is 2000.
-l<val>
Specify the first image we will process. If images 1-13 are
garbage, we can specify -l14.
-m<val>
Search for spot centroids out of <val> different sub-runs of
"spots". If <val> is 4, for example, and there are 120 images in
the FRAMES index, then we will extract spots from images 1, 31,
61, and 91. We'll merge those lists of images together before
auto-indexing and refinement.
-p<val>
Process data images whose filename prefix is <val>. Ordinarily
"xgenproc" picks the prefix in the data directory for which the
largest number of images is present; this option allows us to
specify exactly which prefix to use. Thus if george1a.0001 -
george1a.0180 and george1b.0137 - george1b.0219 are both present
in a directory, we can tell "xgenproc" to work with the george1b
data by specifying xgenproc -pgeorge1b
-w<val>
In multi-centroid mode (-m<val> specified), this specifies the
number of images to be examined within each spots run. The
default value of 1 should probably be used unless the data come
from a small-molecule crystal.
-x<val>
Specify the main-beam X position IN PIXELS as <val>.
-y<val>
Specify the main-beam Y position IN PIXELS as <val>. These
options must be specified together. If they are, the script will
run "calibrate -j" with the appropriate beam-center values
before beginning processing.
-z<val>
Specify the polarization of the input radiation: (1): unpolar-
ized; (2) graphite monochromator; (3): fully polarized, with the
rotation axis in the polarization plane; (4) fully polarized,
with the rotation axis perpendicular to the polarization plane;
(5) like (3) but 95% polarized rather than 100%; (6) like (4)
but 95% polarized.
[datadir]
If a non-flag argument is specified, it is taken to be the name
of the directory in which the search for images is to be per-
formed. By default we specify the current working directory.
This facility is available in part to allow "xgenproc" to be run
from a Python or Motif GUI whose concept of "current working
directory" may be muddy.
EXAMPLES
xgenproc
Process all the images in for the dataset whose prefix is most
plentiful in the current working directory.
xgenproc -d -pgeorge1a -h180 -x2028 -y2029
Process the images with
'LICENSE' for information on usage and redistribution of this file, and
for a DISCLAIMER OF ALL WARRANTIES
Version 5.5.5 October 2005 X-GEN(1)
"get my man!" .. in association with
man2html
and cor