get another manpage




NAME

       X-GEN - xgenproc.py


SYNOPSIS

       xgenproc [--help] [--version] [-dknsuv] [-cfhijlmpwxyz<val>] [datadir]


DESCRIPTION

       "xgenproc.py"  is  a  Python script that allows you to process a single
       run of diffraction data through X-GEN, beginning with  raw  images  and
       culminating  with  the  production  of  several  ascii files containing
       merged, scaled intensity measuremnts.  "xgenproc.py"  has  been  tested
       primarily on data obtained from the two beamlines of Southeast Regional
       Collaborative Access Team, but it should work with  data  collected  on
       home sources and from other beamlines too.

       The  `xgenproc.py'  script is found in the executables directory within
       which the rest of the X-GEN executables live.  On  most  machines  that
       directory will either be /usr/bin itself, or it will be /opt/xgen/exec,
       or /home/[INSTALLER]/xgen/exec, where [INSTALLER] is  the  username  of
       the  person  who  installed  X-GEN  on  your  system.  The executables,
       libraries, and resource files for X-GEN, along with the script  itself,
       are  available for download from http://xgen.iit.edu. Of course, to run
       this, you will also need Python.  This  script  has  been  tested  with
       Python 2.3.5, and it probably runs on later versions; it may be compat-
       ible with earlier versions too.

       If you have collected data into a known directory, you should  be  able
       to  use  this script in a simple manner. First, set your working direc-
       tory to the directory where the data are, e.g.

              cd /u1/ID7/03_18_05_uXX/Fred/uglyxtal

       Then if the data are from the SER-CAT ID line  and  were  collected  in
       omega,  you can process the data by typing "xgenproc.py -s". The script
       will work properly with data from most other sites (including the  SER-
       CAT  ID collected in phi) if you simply type "xgenproc.py". The various
       run-time parameters will enable you to tailor your run  of  xgenproc.py
       to the specific circumstances under which you're running it.

       Frequently asked questions:

       + How do I know if it has worked?

       The  processing  results will be written to a subdirectory of your cur-
       rent directory called xgen. If that directory already  exists  (because
       you  or  someone  else  has already processed these data before, or has
       processed a different dataset in the same directory), then  they'll  be
       written  to the directory xgena. If xgena exists, they'll be written to
       xgenb, and so on up through xgenz.

       So if the output to your screen looks  reasonable  (the  Rmerge  values
       that  you  see  near the bottom are good), then the processing probably
       worked. If you cd to the processing directory (xgen, or xgena, or what-
       ever),  then  if you look at the logfile, you can see how effective the
       processing was. The logfile has a name that ends in .xlg.:

              % cd xgena
              % vi *.xlg

       Examine that .xlg file to see the quality  of  the  data.  The  scaling
       results will be near the end.

       + What if that doesn't work?

       There are several possible reasons it might not work.

       *      The  software may have failed to find the detector images in the
              current directory. If that happens, you may want to specify your
              image    prefixes    explicitly.    Thus   if   your   data   in
              /u1/ID7/03-18_05/UXX/Fred/uglyxtal/ have filenames  ugly01.0001,
              ugly01.0002, . . . , up through ugly01.0137, then you might want
              to try

              % xgenproc.py -s -pugly01

              If it's still unable to find the images, you may need additional
              help from the author (howard@iit.edu).

       *      It  may fail during auto-indexing.  There are some special cases
              for this problem:</li>

              - It may have gotten the stepsize wrong.

              Note the rules I've given above. If you collect on the  bending-
              magnet  beamline,  or if you collect on the ID line using phi as
              your data collection motor, then you don't need to flip the sign
              of  the  stepsize.  If you collect on the ID line using omega as
              your data collection motor, then you need to specify  the  sign-
              flip:

              % xgenproc.py -s

              -  It  may  not have found enough reasonable centroids to refine
              on.

              You can circumvent this by specifying a larger minimum number of
              centroids.  By default the program searches for spots over seven
              images or until it finds 400 centroids,  whichever  comes  last.
              If 400 isn't enough, specify a larger number, e.g.

              % xgenproc.py -c700

              on the BM line or on the ID line with phi data, or

              % xgenproc.py -s -c700

              on the ID line with omega data.

              -  The crystal may be twinned or pathological in some other way.
              International Tables number.  Thus if your crystal is in  space-
              group  P6(2)22  (International Tables # 180), you can specify it
              at runtime as:

              % xgenproc -i180

              You can combine this flag with the others listed aobve if appro-
              priate.

       *      It  may integrate data satisfactorily at first, but fail farther
              along, with rapidly increasing RMS errors in X, Y, or  omega  in
              the  on-the-fly  refinements.  This sometimes indicates that the
              crystal has moved discontinuously somewhere along  the  line  in
              data  collection.  If  the  failure is more gradual, it may be a
              sign of an increase in mosaicity or a problem  with  icing.  You
              may  be  obliged to cut off the integration somewhere before the
              end. You can do this by specifying an upper limit to the  number
              of images to be integrated. Thus if you collected 180 images but
              only the first 130 are well-behaved, you can specify that as

              % xgenproc -h130

              Again, you can combine this flag with others.

       *      It may  integrate  satisfactorily,  but  produce  unsatisfactory
              R(merge)  values  in scaling. There are multiple possible causes
              for this too:

              - The beam-center values are wrong.  Look at your .xlg file just
              before the last integration; the output from the refinement seg-
              ment will contain something that looks like "Start omega:   1.00
              chi:   0.00  phi:   0.00 Mainbeam: 1031.0  1041.9" Note that the
              Mainbeam position _in pixels_ is specified here. If the Mainbeam
              values  in  this  window  are seriously off from where we expect
              them to be, then we may need to specify them specifically.  Here
              if  we  know  they  really should be [X,Y] = [1021,1048], we can
              specify those values at run-time with

              % xgenproc -x1021 -y1048

              - The spacegroup is wrong. Typically this  happens  if  we  have
              been  too  aggressive  in assigning a high symmetry when a lower
              one is real.  Figure out the correct one and specify it:

              % xgenproc -i75

              - There are icerings in the data that are polluting the results.

              This  can  be  readily  dealt  with in the specific applications
              within X-GEN, but not in the one-line processing system.

              - The data have been scaled at higher resolution than is  really
              legitimate.   X-GEN  by default analyzes the output of the inte-
              gration step and estimates the plane-spacing at which the  aver-
              age  value of I/sigma falls to 1.3. It sets the resolution limit
              there. That value may be at higher resolution  than  you  really
              believe  is  appropriate.   My contention is that you can always
              throw out some of  your  data  if  there's  a  little  too  much
              present; you can't get it back if you didn't process at a suffi-
              ciently high resolution. If you want to see what the  statistics
              are  like  at  a different resolution cutoff, you can easily get
              those. To do that, change directories to the directory where the
              processing was being done, establish the X-GEN environment vari-
              ables for your project, and run the X-GEN  "stats"  application,
              specifying  the  resolution  limit  that  you  believe is really
              appropriate:

              % cd xgena      # cd to processing directory
              % source *.com  # establish environment variables
              % stats -on 3.1 # Friedel-mates merged to 3.1A

              This tells the application to calculate intensity statistics for
              the  data out to 3.1 Angstroms, separating Friedel mates for the
              R(merge) calculations.

       + Does "xgenproc" have run-time options?

              Yes. They're listed under "Options".

       + But I want to merge multiple runs!

              This is readily possible with X-GEN, but not through the  "xgen-
              proc"  interface.  I'll develop a simple interface that can han-
              dle multiple data runs shortly.

       + I want to see the images!

              You can readily do  so.  Change  directory  to  your  processing
              directory, establish your environment variables, and examine the
              images with "pdisplay":

              % cd xgen      # cd to processing directory
              % source *.com # establish environment variables
              % pdisplay     # examine your images

              "pdisplay" has many options that you can explore.

       + I'm stuck in ways that aren't discussed here. What do I do?

              Read the full documentation for X-GEN, or call or  e-mail  Andy:
              773-368-5067 (cell), 312-567-5881 (IIT), or howard@iit.edu.


OPTIONS

       --help Print this message, and exit.

       --version
              Print  a header indicating the version of X-GEN that is running.

       -d     Compress the images using the "gzip"  algorithm  after  integra-
              tion.


       -s     Flip  the  sign of the rotation stepsize from the value that the
              program intuits. We try to stay in synch with the stepsize  con-
              ventions  for  every  goniostat,  but  if  we get it wrong, this
              allows recovery.

       -u     Same as "--help", i.e. print this message and exit.

       -v     Same as "--version", i.e. print the  X-GEN  version  information
              and exit.

       -c<val>
              Specify  the  number  of  centroids required for refinement.  In
              general the `spots' functionality will search for 300 spots;  if
              it  doesn't  find  that  many in the first seven frames, it will
              continue looking until it finds  300.  Specifying  -c<val>  will
              direct "xgenproc" to look for a different total.

       -f<val>
              Specify the name of a detector image to be used as an example of
              the  names  of  the  remaining  images.  Thus   if   the   image
              george1a.0037  appears  in  the  run, and we want to process the
              data from all the images with names of the  form  george1a.####,
              then specifying -fgeorge1a.0037 will accomplish this.

       -h<val>
              Specify the last image we will process. This enables two differ-
              ent capabilities: First, if we are collecting while  we're  pro-
              cessing, we can use this flag to specify the last frame we actu-
              ally plan to collect, even if at the time we  start  running  X-
              GEN,  that  frame hasn't been collected yet.  Second, if we know
              that images 57-180 are garbage, we can specify -h56 to  indicate
              that we only want to process up through image 56.

       -i<val>
              Specify  an  International  Tables  spacegroup number.  If auto-
              indexing succeeds, and the Laue group associated with the speci-
              fied  spacegroup  number is a plausible solution to the Bravais-
              lattice calculation, then we will impose this spacegroup  number
              during processing. To specify a rhombohedral indexing of a rhom-
              bohedral unit cell, add 100 to the spacegroup; thus R32  indexed
              rhombohedrally is -i255.

       -j<val>
              Remap  the  detector's geometrical irregularites at each refine-
              ment during integration. On the first one, write the results out
              to  disk; otherwise, simply record them in memory for use within
              the application. <val> in this case is the RMS  error  in  (X,Y)
              above  which the remapping will be performed, in ten-thousandths
              of pixels; thus -j3000 corresponds to remapping only if the  RMS
              error is worse than 0.3 pixels. The default value is 2000.

       -l<val>
              Specify  the  first  image  we  will process. If images 1-13 are
              garbage, we can specify -l14.

       -m<val>
              Search for spot centroids out of  <val>  different  sub-runs  of
              "spots". If <val> is 4, for example, and there are 120 images in
              the FRAMES index, then we will extract spots from images 1,  31,
              61,  and  91.  We'll merge those lists of images together before
              auto-indexing and refinement.

       -p<val>
              Process data images whose filename prefix is <val>.   Ordinarily
              "xgenproc"  picks the prefix in the data directory for which the
              largest number of images is present; this option  allows  us  to
              specify  exactly  which  prefix to use.  Thus if george1a.0001 -
              george1a.0180 and george1b.0137 - george1b.0219 are both present
              in a directory, we can tell "xgenproc" to work with the george1b
              data by specifying xgenproc -pgeorge1b

       -w<val>
              In multi-centroid mode (-m<val> specified), this  specifies  the
              number  of  images  to  be  examined within each spots run.  The
              default value of 1 should probably be used unless the data  come
              from a small-molecule crystal.

       -x<val>
              Specify the main-beam X position IN PIXELS as <val>.

       -y<val>
              Specify  the  main-beam  Y  position  IN PIXELS as <val>.  These
              options must be specified together. If they are, the script will
              run  "calibrate  -j"  with  the  appropriate  beam-center values
              before beginning processing.

       -z<val>
              Specify the polarization of the input radiation:  (1):  unpolar-
              ized; (2) graphite monochromator; (3): fully polarized, with the
              rotation axis in the polarization plane;  (4)  fully  polarized,
              with  the rotation axis perpendicular to the polarization plane;
              (5) like (3) but 95% polarized rather than 100%;  (6)  like  (4)
              but 95% polarized.

       [datadir]
              If  a non-flag argument is specified, it is taken to be the name
              of the directory in which the search for images is  to  be  per-
              formed.  By  default  we  specify the current working directory.
              This facility is available in part to allow "xgenproc" to be run
              from  a  Python  or  Motif GUI whose concept of "current working
              directory" may be muddy.


EXAMPLES

       xgenproc
              Process all the images in for the dataset whose prefix  is  most
              plentiful in the current working directory.

       xgenproc -d -pgeorge1a -h180 -x2028 -y2029
              Process the images with

       'LICENSE' for information on usage and redistribution of this file, and
       for a DISCLAIMER OF ALL WARRANTIES



Version 5.5.5                    October 2005                         X-GEN(1)

"get my man!" .. in association with man2html and cor