

GETTING STARTED WITH THE TVIPS TOOLS


The tiff2smv and tvips2smv utilities convert sequences of related
electron diffraction images, or MicroED _sweeps_, recorded by Tietz
Video and Image Processing Systems (TVIPS) cameras to the Super Marty
View (SMV) format. Diffraction data must be recorded in rolling shutter
mode under continuous rotation of the stage. Output SMV images are
directly suitable for further processing in suites such as DIALS,
MOSFLM, or XDS. tiff2smv supports TIFF images written by either EMMENU
or EM_Player, as long as their pixel values can be losslessly
represented as 16-bit unsigned integers. tvips2smv supports all TVIPS
set files written by EMMENU.


Using the programs

    $ tiff2smv -v foo/*.tiff

This will read all the files whose names end in .tiff from the directory
foo. tiff2smv will provide verbose diagnostics on standard output, but
will not write any converted images.

    $ tiff2smv -o "/tmp/bar_##.img" foo/*.tiff

Convert all the .tiff files in directory foo to SMV format. The template
name for the output files, t_##.img, allows for up to 100 images (_i.e._
/tmp/bar_01.img, /tmp/bar_02.img, ..., /tmp/bar_99.img, /tmp/bar_00.img)
to be written before paths are recycled and the the hundred-first
converted image overwrites /tmp/bar_01.img. tiff2smv will prompt before
overwriting files.

    $ tvips2smv -S :16 -d 3600 -r 0.3 -v -o "/tmp/bar_###.img" foo_00?.tvips

Convert the 15 first images stored in TVIPS set files to SMV format.
Depending on the size of the dataset and the version of EMMENU used to
record it, there can be more than one of these files for each dataset,
and they have to be listed on the command line in order of increasing
sequence number; using shell wildcards such as ? and * will usually
ensure this is the case. The distance in the output files will be set to
3600 mm and the oscillation ranges and start angles for each image will
correspond to a tilt rate of 0.3° per second.

Full details are available in the man pages (man tiff2smv).



AVAILABILITY


The software is copyright Howard Hughes Medical Institute, and made
available under a permissive Internet Software Consortium license.
Binary distributions for a limited set of systems are available from
http://www.ccpem.ac.uk/downloads. Source distribution tarballs are
available for download from http://cryoem.ucla.edu/downloads.



COMPILING AND INSTALLING THE PROGRAMS


Dependencies

The TVIPS tools depend on LibTIFF, version 4.0.0 or later. To build the
sources C- and C++-compilers, flex (version 2.5.34 or later), and CMake
(version 3.4 or later) are required. All steps should complete without
errors or warnings. If they do not, please file a report on the gonenlab
issue tracker.


Unix-like systems

Source archives of the TVIPS tools can be unpacked using

    $ tar -xvzf tvips-tools-X.Y.Z.tar.gz

where X, Y, and Z are the major, minor, and patch version numbers, which
change from release to release according to the rules of semantic
versioning. The sources are configured in a fresh build directory with

    $ mkdir build
    $ cd build
    $ cmake ../tvips-tools-X.Y.Z -DCMAKE_INSTALL_PREFIX:PATH=${prefix}

The name of the build directory is arbitrary; the (relative) path to the
source directory, which contains the CMakeLists.txt file
(../tvips-tools-X.Y.Z in the example above), needs to be adjusted
correspondingly. ${prefix} denotes the desired destination location of
the compiled binaries and the accompanying documentation. If LibTIFF is
installed in a non-standard location, ${LibTIFF_prefix}, the cmake
invocation may need to be amended with
-DTIFF_INCLUDE_DIR=${LibTIFF_prefix}/include and
-DTIFF_LIBRARY=${LibTIFF_prefix}/lib/libtiff.so.

Once the build system has been generated, compile the sources using

    $ make

Optionally, run the test suite

    $ make test

Finally, to install the binaries and the documentation, do

    $ make install


MinGW on Microsoft Windows

The TVIPS tools can be compiled on Microsoft Windows using the
Minimalist GNU for Windows (MinGW) development environment. MinGW does
not provide flex, but a Windows-port can be obtained from Win
flex-bison. Since MinGW does also not provide LibTIFF and the binaries
available from G_nu_Win32 are too old, a custom installation is
required. With source code downloaded from
http://download.osgeo.org/libtiff, LibTIFF can be installed from Command
Prompt (cmd.exe) using

    > unzip tiff-4.0.9.zip
    > mkdir tiff-4.0.9-build
    > cd tiff-4.0.9-build
    > cmake -G "MinGW Makefiles" -DCMAKE_INSTALL_PREFIX=%LibTIFF_prefix% ..\tiff-4.0.9
    > mingw32-make
    > mingw32-make test
    > mingw32-make install

where %LibTIFF_prefix% denotes the desired location of the LibTIFF
installation. To build a static LibTIFF without ZIP support, the CMake
invocation can be amended with -DBUILD_SHARED_LIBS=OFF -Dzlib=OFF.

The build system for the TVIPS tools must be made aware of the
installation prefix of the LibTIFF library prior to compilation

    > tar -xvzf tvips-tools-X.Y.Z.tar.gz
    > mkdir build
    > cd build
    > cmake -G "MinGW Makefiles" -DCMAKE_INSTALL_PREFIX:PATH=%prefix%  -DFLEX_EXECUTABLE="%win_flex_bison_prefix%\win_flex" -DTIFF_INCLUDE_DIR="%LibTIFF_prefix%\include" -DTIFF_LIBRARY="%LibTIFF_prefix%\bin\libtiff.dll" ..\tvips-X.Y.Z
    > mingw32-make
    > mingw32-make test
    > mingw32-make install

where %prefix% denotes the desired destination location of the compiled
binaries and the accompanying documentation, and X, Y, and Z are the
major, minor, and patch version numbers of the TVIPS tools.
%win_flex_bison_prefix% is the path to where the Win flex-bison was
extracted.

To run a dynamically linked executable, the runtime linker needs to
locate libtiff.dll, libgcc_s_dw2-1.dll, libiconv-2.dll, and
pthreadGC-3.dll. On Microsoft Windows, this can be accomplished by
placing a copy of these dynamic-link libraries in the same directory as
the compiled executables. The first library is found in
%LibTIFF_prefix%\bin as defined above; the remaining libraries are part
of the MinGW installation and located in C:\MinGW\bin by default. This
does not necessarily violate the GNU General Public License due to the
GCC Runtime Library Exception.

To build statically linked TVIPS tools, add
-DCMAKE_EXE_LINKER_FLAGS=-static and replace bin/libtiff.dll with
lib/libtiff.a in the definition of TIFF_LIBRARY.

The test suite should pass for dynamically and statically linked TVIPS
tools.



DEVELOPERS


Developers wishing to contribute source code will need access to the
live repository, which requires the Git utility, a user account at
GitHub, and membership in the gonenlab group. The last step may require
some interaction with the present developers in the MicroED team;
sending an e-mail to microed@hhmi.org is a good way to start. To obtain
a local copy of the source tree, the GitHub account first has to be set
up for SSH access, which means generating an SSH key and adding it to
the GitHub account. Then

    $ git clone git@github.com:JaneliaSciComp/gonenlab.git mygonenlab

This clones the master branch of the remote GitHub repository to a local
directory. The directory name is arbitrary; mygonenlab is just an
example, and will be used in what follows. The build system currently
requires git to be in PATH, otherwise the compiled tools will report
incorrect information when invoked with the -V or --version option.

To keep the sources up-to-date as other developers push their changes,
do

    $ cd mygonenlab
    $ git pull

Local modifications to the source code may give rise to conflicts in
this step. Such conflicts have to be resolved manually.


Building the documentation

Releases include pre-compiled documentation, in machine-independent man
and portable document format (PDF). Developers who are working on a
local clone of the GitHub repository and wish to have the documentation
installed will have to build it separately. This requires Libxml2,
Apache FOP, and the DocBook XSL stylesheets. To build the man-pages, do

    $ make man

To generate the PDF documentation, do

    $ make pdf

To generate the documentation in HyperText Markup Language (HTML), do

    $ make html


Rolling a distribution

Building a tape archive for distribution requires an uncluttered source
tree. This is because CMake’s packaging mechanism works by collecting
_all_ files in the source directory. Unlike the GNU Autotools, CMake
does not provide a distclean target; developers are therefore encouraged
to keep their source and build directories separate. Then

    $ make dist

This step also requires pandoc to convert the GitHub-flavored markdown
files LICENSE.md and README.md to plain text.

If the source directory is unchanged since it was last tagged, this
creates a file called tvips-tools-X.Y.Z.tar.gz in the build directory.
Otherwise, the name of the distribution tarball will be
tvips-tools-branch-date.tar.gz, where branch is the name of the current
branch and date is the current date in the form %Y%m%d (see
strftime(3)).
