%\VignetteIndexEntry{rbamtools}
%\VignettePackage{rbamtools}
\documentclass[11pt,a4paper]{article}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Load Packages
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\usepackage{amsmath}                        % align environment

\usepackage{helvet}                         % Actually switches the default
                                            % sans serif font to Nimbus Sans L
\usepackage{courier}

\usepackage[ngerman]{babel}
\usepackage[utf8]{inputenc}                 % utf8 umlaut

\usepackage{booktabs}                       % Table-Style
\usepackage{url}

\usepackage{makeidx}                        % Creation of index

\usepackage[usenames,dvipsnames]{color}
\usepackage{sectsty}
\usepackage{hyperref}                       % Index links, should be loaded as
                                            % last package (except geometry)


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Place settings
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\allsectionsfont{\sffamily\color{RoyalBlue}}

% Switches to standard sans serif for everything but math mode
\renewcommand{\familydefault}{\sfdefault}

% Paragraph
\parindent0mm

\newcommand{\code}[1]{{\texttt{#1}}}
\newcommand{\rtx}[1]{{\textsf{#1}}}
\newcommand{\rcl}[1]{{\texttt{#1}}}
\newcommand{\rfc}[1]{{\texttt{#1}}}
\newcommand{\robj}[1]{{\texttt{#1}}}
\newcommand{\rarg}[1]{{\texttt{#1}}}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Title & index
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\title{Using the rbambools package}
\author{Wolfgang Kaisers, CBiBs HHU Dusseldorf}
\date{\today}


\begin{document}
\maketitle
\SweaveOpts{concordance=TRUE}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Prepare workspace
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

<<echo=FALSE>>=
library(rbamtools)
library(xtable)
options(width=60)
@

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\section{Introduction}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

BAM files are an important and powerful file format in Bioinformatics. This
package pursues several objectives:
\begin{itemize}
    \item Provide a technical (reading and writing) access to BAM files from
        within R.
    \item Give an authentic representation of the informational structure inside
        BAM files as programming interface.
    \item Provide a fast, C-based access to special (cumulative) aspects of the
        stored information.
\end{itemize}
%
These objectives transform into three implementational layers:
\begin{itemize}
    \item The samtools C-library (written by Heng Li).
    \item C-based alignment and align-gap and gap-sites container.
    \item A R S4 class library.
\end{itemize}
The samtools library is an adapted version of samtools. Samtools version 0.1.18
(last modified 02 Sept 2011) had been downloaded on 07 Sept 2011 from the
samtools homepage
\footnote{\url{http://sourceforge.net/projects/samtools/files/samtools/0.1.18/}}.
The then current version of the samtools file format description was
(v1.4-r985, 0.1.18).\\

All file interactions are done via samtools. There is C-code which handle alignment
data for whole ranges and C-code for accumulation of information about
splice-sites from gapped alignments. Samtools and container C-structures are
connected to S4 objects by external pointers (EXTPTRSXP)\\

The R-part of the source code calls C-functions which directly communicate
with the samtools library.

Align-gaps are emphasized here because they are highly informative
representations of genomic splice-sites in RNA-seq data.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% SAM file format
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\section{SAM file format}

Data in BAM files is compressed and optionally indexed data in SAM file format.
The current definition of the SAM file format~\cite{sam} can be found on the
samtools homepage\footnote{\url{http://samtools.sourceforge.net/SAM1.pdf}}.\\

BAM files contain sequence alignment data which is the result of potentially
incomplete matching sequence snippets to a reference sequence. In practice the
snippets are DNA sequences which come from short read sequencing of DNA or RNA
extracted from a biological sample and the reference sequence is a
genome reference.\\

Usually one BAM file contains alignments data from one biological sample where the
read number is in range of 10 to 100 million. The size of the
corresponding compressed files usually is 1 to 15 Gbyte. A very important
feature of BAM files is that sorted BAM files can be indexed and indexed files
allow random access. This allows very fast access to alignments that are located in
arbitrary regions of the reference genome.\\

The content of BAM files is divided in a header section and an alignment section.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{The header section}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

The header section contains the following information:\\
\newline
\begin{tabular}{l l l}
  \hline
    Tag & Description                    & Explanation\\
  \hline
    \hline
    HD  & Header line                    & Format version and sorting\\
    SQ  & Reference sequence dictionary  & Indexed reference sequences*\\
    RG  & Read group                     & Sequencing technology\\
    PG  & Program                        & Alignment program\\
    CO  & Comment                        & \\
  \hline
  \multicolumn{3}{p{11.5cm}}{\scriptsize *Entries in the reference sequence
  dictionary usually are Chromosomes (e.g. 'chr1')}
\end{tabular}\\
\vspace{1 em}
\\
There are accessor functions in this package for reading and writing the listed
fields. The header section is stored and retrieved as binary structure
(\rcl{bamHeader}) which is converted into a tag delimited string representation
(\rcl{bamHeaderText}).
All processing steps on BAM-header data work on the string representation.
\rcl{rbamtools}-objects parse and compose strings from and to object slots which
then can be accessed via script code.\\

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{The reference sequence dictionary}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

The reference sequence dictionary section contains a list of reference sequences
(usually chromosomes). Only two of six fields (declared in the SAM file format
specification) usually only two are used:\\
\begin{center}
\begin{tabular}{ll}
  \hline
    Tag & Description\\
  \hline
    SN  & Reference sequence name\\
    LN  & Reference sequence length\\
  \hline
\end{tabular}
\end{center}
\vspace{1 em}
The reference sequence dictionary section misses an index entry (refid) which
is used in alignment structures and is described below in ~\ref{sbsec:refid}.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{The alignment section}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

The align section contains a series of alignments datasets. Each alignment describes
the coordinates of the identified sequence matches in the reference sequence.
The information for each alignment basically consists of:\\
\begin{center}
\begin{tabular}{l l}
  \hline
    Field & Content\\
  \hline
    QNAME & alignment name (read identifier)\\
    RNAME & Reference sequence identifier\\
    POS   & Mapping position: \underline{0-based}\\
    CIGAR & Matching type string\\
    FLAG  & A set of bitwise flags.\\
  \hline
\end{tabular}
\end{center}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{The RNAME identifier: refid} \label{sbsec:refid}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Although RNAME associates with a textual entry, usually this field contains a
number which identifies a sequence in the header section. To make things
complicated, RNAME is a \textit{0-based} sequential identifier which is not
explicitly included in the \textit{Reference sequence dictionary} (SQ). So,
RNAME=0 means the first SQ entry and the '0' is not present in the header.\\
%
We call this missing value \textit{refid} throughout this document and there are
functions in this package that automatically generate and use this id. The refid
value is used by the \texttt{samtools} library as sequence identifier in
alignment structures and for defining ranges in index based random access.\\

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Position}\label{sbec:pos}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The position entry gives the alignment start position. For checking the
similarity between query and reference sequence it may be feasible to
compare the sequences manually in single cases.\\

In order to find the exact matching position it's necessary to notice the base
of the position notation. We distinguish \textit{0-based} and \textit{1-based}
position notations. They differ by the index of the starting position
(and therefore all positions).\\

The first position in a \textit{0-based} notation is 0 whereas the first
position in a \textit{1-based} notation is 1:\\
\begin{center}
\begin{tabular}{|l|c|c|c|}
  \hline
    0-based & 0 & 1 & 2\\
  \hline
    1-based & 1 & 2 & 3\\
  \hline
\end{tabular}
\end{center}
%
Both notations appear in samtools. The SAM file format specification says (see
~\cite{sam}, section 1.4): 'POS: 1-based leftmost mapping POSition of the first
machting base'"'. Samtools source code comments (bam.h, line 164) state the
contrary: 'pos 0-based leftmost coordinate'. Experiences with aligners
(tophat 2.0.0) and annotation data (Ensembl and UCSC) indicate that
the latter seems to be true (i.e. position entries are 0-based).\\
%
In order to reflect the technical file content, two functions (\rfc{position}
on \rcl{bamAlign} objects and \rfc{as.data.frame} on \rcl{bamRange} objects)
return the file contained value (which is 0-based). In order to get values that
are congruent with annotation (and IGV genome-browser data) or positions
in DNAStringSet (Bioconductor Biostrings package) objects, the position values
have to be increased by one.\\
%
The \rcl{bamGapList} objects which operate on alignment gaps contain
\textit{1-based} positions. So, overlapping with annotation data,
can be done without correction.\\

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Navigation on reference sequence}\label{sbec:dir}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

Printing the reference sequence results in characters that are ordered from left
to right in ascending order of their position coordinate (consistent with
ordinary reading succession). We refer to this image when two or more locations
are compared. Lower coordinates are assumed to be on the \textit{left} side and
higher coordinates are assumed to be on the \textit{right} side.\\
%
So, for genes coding on the (+) strand, \textit{left} would be synonymous to
\textit{upstream} and \textit{right} would be synonymous to \textit{downstream}.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{CIGAR string}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Alignment algorithms usually tolerate to some extend inexact matching. The type
of matching is described in the CIGAR string. For a complete list of CIGAR
operations see ~\cite{sam} 1.4, Nr. 6.
The CIGAR string is made up of CIGAR-items. A CIGAR-item consists of a integer
number and a character. The number counts the affected positions (cigar-length).
The character describes the type of operation (cigar-type). The following table
shows relevant operations:\\
\newline
\begin{tabular} {lll}
  \hline
    Operation & Label         & Description\\
  \hline
    M         & Match         & Exact match of x positions\\
    N         & Alignment gap & Next x positions on ref don't match\\
    D         & Deletion      & Next x positions on ref don't match\\
    I         & Insertion     & Next x positions on query don't match\\
  \hline
              &               & (x = cigar-length)\\
\end{tabular}
\vspace{1 em}
\newline
The operations 'N' and 'D' are mechanistic identical but they describe
biological different entities: 'D' means genomic deletions, where few
nucleotides on the genome get lost whereas 'N' means gaps which occur in
RNA-seq alignments. These gaps are due to DNA-splicing events and their size can
achieve  magnitude of $10^3 - 10^5$.\\
%
\paragraph{First example:} The shown alignment is an exact match and will give
\texttt{position = 2} (0-based!) and \texttt{CIGAR = 6M}:\\
\newline
\verb|AAGTCTAGAA (ref)|\\
\verb|  GTCTAG   (query)|\\
%
\paragraph{Second example:} We consider an alignment with two nucleotides ("GA")
inserted into the reference. The alignment entries will be \texttt{position=3}
(0-based!) and \texttt{CIGAR=3M2I2M}:\\
\newline%
\verb|AAAGTCGATGAA (ref)|\\
\verb|   GTC  TG   (query)|\\
%
\paragraph{Third example:} The next exampls contains a deletion on the
reference. The 'C' in the query sequence has no match.
The alignment entries will be \texttt{position=3}
and \texttt{CIGAR=2M1D3M}:\\
\newline%
\verb|AAGT TAGAA (ref)|\\
\verb|  GTCTAG   (query)|\\
%
\paragraph{Fourth example:} This is a gapped alignment (due to a splicing event
in RNA-seq). It will give the entries \texttt{position=3} and
\texttt{CIGAR=3M7N4M}:\\
\newline%
\verb|CCCTACGTCCCAGTCAC (ref)|\\
\verb|   TAC       TCAC (query)|\\
\newline%
We see the alignment gap (GTCCCAG). From the 'GT' and 'AG' at the gap
boundaries, it can be assumed that this splice-site is on the (+) strand.
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Gapped alignments}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
A special focus of functionality inside this package are Alignment gaps.
Alignment gaps in RNA-seq experiments are viewed as phenomenons that rely on
biological splicing mechanisms during protein-biosynthesis and the resulting
exon-intron structure of the genome.

\setlength{\unitlength}{1cm}
\begin{picture}(6,4)

  % ref vector
  \put(1,0.5){\vector(1,0){8}}
  % ref text
  \put(1,0){CTCCGACAGACAGGACACCAGCTGTATGGCCG}

  % Exon boxes
  \put(1,1){\framebox(2,1){left exon}}
  \put(7,1){\framebox(2,1){right exon}}

  % connective lines
  \put(3,2){\line(4,1){2}}
  \put(5,2.5){\line(4,-1){2}}
\end{picture}
\vspace{1 em}
\newline

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Gap-sites}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Gap-sites are alignment gaps (=gap-regions) that are shared by one or more
alignments. The nucleotides on the reference sequence that are skipped in the
alignment (i.e. the reference region which is depicted by "N" cigar items)
form the gap-region.\\
Gap-sites are also characterized by the fact that they are bordered by
M-segments on either side. The amount of information about the existence of
gap-site in the alignment is proportional to the number of matching nucleotides
that make up the framing M-segments. The calculated derived values on gap-sites
therefore center on three measures:\\
%
\begin{itemize}
    \item The number of alignments that define the gap-site.
    \item The Length of the framing M-segments.
    \item The number of different length values in the framing M-segments.
    \item The number of alignments (probes, number of BAM-files) in which the
        gap-site is found.
\end{itemize}
\vspace{1 em}
%
Gap-sites are of special interest in RNA-seq experiments because they arise
from mRNA sequence which spans a processed splice site (splicing results in
removal of intronic sequence ranges from pre-mRNA). Gapped alignments contain
highly specific information about splicing events. One of the  central
objectives in RNA-seq experiments is the identification and
quantification of splice events.\\
\newline
In order to describe and illustrate the parameters that are calculated and kept
within this package we show the following example:

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Example}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The following table example shows a short reference sequence and three different
alignments that define a gap-site. The reference nucleotides that consitute the
gap-region are printed in red:\\
\begin{center}
\begin{tabular}{l|l|l|l}
                                  & qname        & position & CIGAR \\
    \hline
        \verb|   AG      CCTTGATG   | & align1       & 3        & 2M6N8M \\
        \verb|  CAG      CCTTGAT    | & align2       & 2        & 3M6N7M \\
        \verb| CCAG      CCT        | & align3       & 1        & 4M6N3M \\
    \hline
        \texttt{CCCAG\textcolor{red}{GTCCAG}CCTTGATGTCC} & (reference)
                                                    & (0-based) &\\
\end{tabular}
\end{center}
\vspace{1 em}
%
For each gapped alignments from which the gap-site is constituted, three values
concerning the number of matching nucleotides are kept:
\begin{itemize}
  \item \textbf{lcl} (left cigar length) is the length of the left adjacent
        match in the CIGAR string.
  \item \textbf{rcl} (right cigar length) is the length of the right adjacent
        match in the CIGAR string.
  \item \textbf{mcl} (minimum cigar length) is the minimum of the \texttt{lcl}
        and \texttt{rcl} value for each alignments.
\end{itemize}

For these parameters we have values in the example:\\
\begin{center}
\begin{tabular}{l|l|l||r|r|r}
    qname        & position & CIGAR  & lcl & rcl & mcl\\
  \hline
    align1       & 3        & 2M6N8M & 2   & 8   & 2  \\
    align2       & 2        & 3M6N7M & 3   & 7   & 3  \\
    align3       & 1        & 4M6N3M & 4   & 3   & 3  \\
  \hline
\end{tabular}
\end{center}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Gap-site coordinates}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
For each gap-site, \texttt{localisation-coordinates} are defines as:
\begin{itemize}
    \item \textbf{refid} (reference sequence identifier)
    \item \textbf{lend} (left-end) is the (1-based) coordinate of the last
        matching nucleotide on the left side:
        \texttt{CCCA\textcolor{green}{\large{G}}\textcolor{red}{GTCCAG}CCTTGATGTCC}
    %
    \item \textbf{rstart} (right-start) is the (1-based) coordinate of first
        matching nucleotide on the right side:
            \texttt{CCCAG\textcolor{red}{GTCCAG}\textcolor{green}{\large{C}}CTTGATGTCC}
    %
\end{itemize}
\vspace{1 em}
We call alignments sharing identical gap-localisation-coordintates
\texttt{gap-site-defining-aligns}. In order to derive a lower boundary for the
size of the adjacent exons are calculated:
\begin{itemize}
    \item \textbf{lstart} (left-start) is the (1-based) coordinate of the
        leftmost nucleotide for which a match exsists in the set of left
        adjacent matching regions:\\
        \texttt{C\textcolor{green}{\large{C}}CAG\textcolor{red}{GTCCAG}CCTTGATGTCC}.\\
        The position is calculated by $lstart=lend-max(lcl)+1$.
    %
    \item \textbf{rend} (right-end) is the (1-based) coordinate of the rightmost
        nucleotide for which a match exists in the set of right adjacent
        matching regions:\\
        \texttt{CCCAG\textcolor{red}{GTCCAG}CCTTGAT\textcolor{green}{\large{G}}TCC}.\\
        The position is calculated by $rend=rstart+max(rcl)-1$.
    %
\end{itemize}
As derivative, the number of nucleotides in the gap-region (denoted
\textbf{gaplen}) is calculated as $gaplen=rend-lstart-1$.
Alltogether, the gap-site and the adjacent putative matching regions in this
example are:\\
%
\begin{center}
\texttt{C\textcolor{blue}{\large{CCAG}}\textcolor{red}{GTCCAG}\textcolor{blue}{\large{CCTTGATG}}CCTTGATGTCC}.\\
\end{center}
%
The associated numeric values for the shown example are:\\
\begin{center}
\begin{tabular}{lrrl}
  \hline
      Name           & value & base & \\
  \hline
      refid          &  0    &    0 & We assume, there is only one reference sequence\\
      lstart         &  2    &    1 & Leftmost match position (\textcolor{blue}{C})\\
      lend           &  5    &    1 & Last match on left side  (\textcolor{blue}{G})\\
      rstart         & 12    &    1 & First match on right side  (\textcolor{blue}{C})\\
      rend           & 20    &    1 & Rightmost match position  (\textcolor{blue}{G})\\
      gaplen         &  6    &      & Number of nucleotides in gap\\
  \hline
\end{tabular}
\end{center}
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Quantification of alignment numbers}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The number of gap-site-defining-aligns are quantified in:
\begin{itemize}
    \item \textbf{nAligns}, the number of alignments that define the gap-site.
    \item \textbf{nProbes}, the number of alignments (BAM-files) in which this
        gap-site is found.
\end{itemize}
%
In the present example, the resulting values are nAligns=3 and nProbes=1.
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Quantification of informational support for gap-site's}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
In order to quantify the information content for each gap-site \texttt{lcl} and
\texttt{mcl} values are stored as single byte values inside of an unsigned
long long integer. We define \texttt{n} as the number of bytes they contain.\\
\\%
On a 32-bit operating system there is $n=4$ and on a 64-bit operating system
$n=8$. With that, we van view \texttt{lcl} and \texttt{mcl} as n-dimensional
vectors: $lcl=(lcl_i)_{i=1,\ldots,n}$ and $mcl=(mcl_i)_{i=1,\ldots,n}$ in which
values are placed in descending order.

\begin{itemize}
    \item \textbf{nlstart}, the number of different match start positions,
        which equals the number of different values in the \texttt{lcl}
        vector.\\
        $nlstart := \# \{lgl_i : i=1,\ldots,n\}.$
    %
    \item \textbf{lm\_sum}, the number of matching nucleotides on the left side
        of the gap.\\
        $lm\_sum := \sum_{i=1}^ lgl_i$.
    %
    \item \textbf{qsm}, the sum of the four largest \texttt{mcl} values
        (quartet sum of minimal cigar length): $\sum_{i=1}^4 mcl_i$
\end{itemize}

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Gap quality score (gqs)}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The gap quality score is calculated as
\begin{align}
    gqs &= 10   \times \frac{nlstart}{n} \quad \frac{2 qsm}{4}\\
        &= 10   \times \frac{\# \{lgl_i : i=1,\ldots,n\}}{n}
                \times \frac{2 \sum_{i=1}^4 mcl_i}{4}
\end{align}
The score quantifies number of alignment start positions and matching nucleotides in
order to distinguish biological existing splice-sites from alignment
phenomenons.\\
\\
The stored information accumulates with increasing the number of included
alignments (BAM-files). The score is given as a positive integer value and the
maximum reachable number is 10 * read-length.\\
\\
The higher the score the more likely is the fact that a gap-site represents a
splice-site. gqs is not intended quantify gene expression although the two
values correlate.


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Object Types
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\section{Object types inside rbamtools package}

The description of object types in this section starts with reading and writing
access to BAM files, proceeds to objects which elementary data inside BAM files
and ends with the description of more complex containers.
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Included example files within rbamtools}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
There are two example files included which are located in the '/inst/extdata'
subdirectory of the package installation site. The directory contains a sorted
BAM file 'accepted\_hits.bam' and the corresponding index file
'accepted\_hits.bam.bai'.

They were produced (using the \rfc{extractRanges} function) from a RNA-seq
experiment. A human probe was sequenced using an Illumina Hiseq sequencer.
Fastq-reads were aligned with tophat against homo sapiens UCSC reference genome.
Complex alignments (i.e. alignments with nCigar > 1) were extracted for genes KLHL17
(chr1) and SNRNP25 (chr16). The BAM file contains 3333 alignments.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Reading and writing access}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Immediate reading and writing access is provided by \rcl{bamReader} and
\rcl{bamWriter} Objects.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{bamReader}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
An object of class bamReader is constructed and returnd by the function
\rcl{bamReader} in the following way:
<<echo=TRUE>>=
bam <- system.file("extdata",
                "accepted_hits.bam", package="rbamtools")
# Open bam file
reader <- bamReader(bam)
@
%
An opened bamReader can be used to access the BAM header section and to read
alignments sequenitally. \rcl{bamReader} can also be used to sort and index BAM
files.\\
%
Sorting large BAM files requires some time and produces intermediate files. So
the recommended way of sorting large BAM files is to use the samtools command
line version. Sorting BAM files within R can be done with:
%
<<eval=FALSE>>=
bamSort(reader, prefix="my_sorted",
            byName=FALSE, maxmem=1e+9)
@
%
Sorted BAM files can be indexed. Indexing results in a second file which is
usually named as the BAM file itself with an added suffix ".bai". An index file
can be created with:
%
<<eval=FALSE>>=
createIndex(reader, idx_filename="index_file_name.bai")
@
%
Omitting the \verb|idx_filename| argument results in adding the ".bai" suffix
to the filename of the BAM file which is then automatically located in the
same directory as the BAM file itself:
%
<<eval=FALSE>>=
createIndex(reader)
@
The creation of indexes for large BAM files (10 GB) takes some minutes time but
can readily be done with this routine and of course has to be done only once per
file.\\
\\
The index files must be loaded before they can be used:
<<echo=TRUE>>=
idx <- system.file("extdata", "accepted_hits.bam.bai", package="rbamtools")
loadIndex(reader, idx)
@
The reader object can be checked for for loaded index with:
<<echo=TRUE>>=
indexInitialized(reader)
@
A shortcut for opening a BAM file and loading the \textit{standard} index at the same
time is:
<<echo=TRUE>>=
reader <- bamReader(bam, idx=TRUE)
@
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Tabled reference sequences: \rcl{getRefData}}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
A data.frame with the reference sequences contained in the BAM header can be
obtained with:
<<echo=TRUE>>=
getRefData(reader)
@
The returned data.frame contains in the first column (ID) the mentioned
refid~\ref{sbsec:refid} value which is not part of the header but uses as
identifier for alignments and ranges.
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{bamWriter}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
For creation of a \rcl{bamWriter} object, a \rcl{bamHeader} and a filename must
be given. The most convenient way of obaining a \rcl{bamHeader} class is
to obtain one from an opened \rcl{bamReader} object.
%
<<eval=FALSE>>=
header <- getHeader(reader)
writer <- bamWriter(header,"test.bam")
# Write alignments using bamSave
bamClose(writer)
@
%
alignments can be written to a BAM file either from single instances of
\rcl{bamAlign}'s or from whole \rcl{bamRange} objects.


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Elementary data structures
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\section{Elementary data structures}

The content of BAM files can be divided in \texttt{header} section and
\texttt{alignment} section.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Structures for header section}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The complete header information (in binary representation) can be retrieved from
a BAM file with the function \texttt{getHeader}. An object of this type is
needed for creation of a \texttt{bamWriter} object.\\
\\
In order to get Access to the data itself, the binary data has to be converted
into a string representation which is maintained inside an object of class
\texttt{bamHeaderText}:
%
<<echo=TRUE>>=
header <- getHeader(reader)
htxt <- getHeaderText(header)
@
%
The header section is divided into several seqments (as described above) with
data tags that describe the origin of the contained alignments.
For each segment there is a class which can be be obtained by calling the
appropriate function on a \texttt{bamHeaderText} object:\\
\newline
\begin{tabular}{l l l l}
  \hline
    Segment & Description & S4 class & Accessor\\
  \hline
    HD & The header line & headerLine & headerLine\\
    SQ & Reference sequence dictionary & refSeqDict & refSeqDict\\
    RG & Read group & & \\
    PG & Program & headerProgram & header Program\\
    CO & Comment & & \\
  \hline
\end{tabular}
\vspace{1 em}
\newline
A complete \rcl{bamHeader} object can be created from scratch with the following
code:\\
<<>>=
bh <- new("bamHeaderText")

headl <- new("headerLine")
setVal(headl, "SO", "coordinate")

dict <- new("refSeqDict")
addSeq(dict, SN="chr1",  LN=249250621)
addSeq(dict, SN="chr16", LN=90354753)
dict

prog <- new("headerProgram")
setVal(prog, "ID", "TopHat")
setVal(prog, "PN", "tophat")
setVal(prog, "CL",
    "tophat --library-type fr-unstranded hs_ucsc_index reads.fastq")
setVal(prog, "DS", "Description")
setVal(prog, "VN", "2.0.0")
bh <- bamHeaderText(head=headl, dict=dict, prog=prog)
header <- bamHeader(bh)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Structures for alignment section}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Single alignments can be retreaved from opened reader via \rfc{getNextAlign}:
<<>>=
align <- getNextAlign(reader)
@


The alignment section in BAM files is a series of alignment (align) records.
The data inside of each record is represented by a \texttt{bamAlign} object.
Section 1.4 ~\cite{sam} describes the information content for each alignments in
detail. The fields and the corresponding \rcl{bamAlign} accessors are listed
below:\\
\begin{center}
\begin{tabular}{lll}
  \hline
    Field & Description & Accessor\\
  \hline
    QNAME & Name                        & name\\
    FLAG  & Multiple Flags              & flag\\
    RNAME & refid                       & ~\ref{sbsec:refid} refID\\
    POS   & Mapping position            &  ~\ref{sbec:pos} position (0-based!)\\
    MAPQ  & Mapping quality             & mapQuality\\
    CIGAR & CIGAR string                & cigarData\\
          & Number of cigar entries     & nCigar\\
    RNEXT & Ref name of mate segment    & mateRefID\\
    PNEXT & Position of mate segment    & matePosition\\
    SEQ   & segment sequence            & alignSeq\\
    QUAL  & Pred-scaled Quality String  & alignQual\\
  \hline\\
\end{tabular}
\end{center}
The accessors can be used in the following way:
<<eval=FALSE>>=
name(align)
flag(align)
refID(align)
position(align)
mapQuality(align)
cigarData(align)
nCigar(align)
mateRefID(align)
matePosition(align)
alignSeq(align)
alignQual(align)
@
%
%
\paragraph{Flag segments:} The flag field contains multiple bit-coded flags
which are kept together inside an integer value:\\
\begin{center}
\begin{tabular}{lll}
  \hline
    Bit & Description & Accessor\\
  \hline
    0x1   & Paired alignments & paired\\
    0x2   & Proper pair & properPair\\
    0x4   & Unmapped & unmapped\\
    0x8   & Mate umapped & mateUnmapped\\
    0x10  & Reverse Strand & reverseStrand\\
    0x20  & Mate reverse Strand & mateReverseStrand\\
    0x40  & First in pair & firstInPair\\
    0x80  & Second in pair & secondInPair\\
    0x100 & Secondary align & secondaryAlign\\
    0x200 & Not passing quality control & failedQC\\
    0x400 & PCR or optical duplicate & \verb|pcrORopt_duplicate|\\
  \hline
\end{tabular}
\end{center}
%
The following code demonstrates the usage of the flag-accessors:
<<eval=FALSE>>=
paired(align)
properPair(align)
unmapped(align)
mateUnmapped(align)
reverseStrand(align)
mateReverseStrand(align)
firstInPair(align)
secondInPair(align)
secondaryAlign(align)
failedQC(align)
pcrORopt_duplicate(align)
@
%
%
The same accessors can also be used for setting values:
<<>>=
unmapped(align) <- TRUE
@
%
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Creating bamAlign objects from scratch}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The {bamAlign} function can be used to create \rcl{bamAlign}
objects from scratch:
<<>>=
align <- bamAlign("HWUSI-0001", "ATGTACGTCG", "Qual/Strng",
                "4M10N6M", refid=0, position=100)
align
name(align)
alignSeq(align)
alignQual(align)
cigarData(align)
refID(align)
position(align)
@
%
The created bamAlign objects can be added to a \rcl{bamRange} list or be
written to a BAM-file via \rcl{bamWriter}.


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Complex and cumulative container
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\section{Complex and cumulative container}


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{alignments lists for specific reference regions: bamRange}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\rcl{bamRange} objects manage a list of \rcl{bamAlign}'s. As BAM files usually
contain alignment results against a reference-genome, \rcl{bamRange} objects
contain list of all alignments that match between a given start and stop position on
a given chromosome. Region coordinates are thereby defined by a
refid~\ref{sbsec:refid} and a start and stop position.


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Reading bamRange from bamReader}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
In order to create a bamRange object, an index-initialized \rcl{bamReader}
object and a numeric coordinate-vector of length three are passed to the
\rcl{bamRange} function.\\
\\
There are several ways to provide genomic coordinates from which the alignments
shall be retrieved.
The first way is to specify a circumscribed genomic region
(e.g. where a gene of interest is located). The names for the coordinates are
not required and only added for explanational purposes:
%
<<echo=TRUE>>=
coords <- c(0, 899000, 900000)
names(coords) <- c("refid","start","stop")
range <- bamRange(reader,coords)
size(range)
@
%
The second way is to specify coordinates for a whole reference sequence
(chromosome). As can be seen from the output of the \rcl{getRefData} function,
the coordinates for the whole first chromosome should be given as:
%
<<echo=TRUE>>=
getRefData(reader)
coords <- c(0,0,249250621)
names(coords) <- c("refid","start","stop")
range <- bamRange(reader,coords)
size(range)
@
%
The function \rcl{getRefCoords} is used here as shortcut:
%
<<echo=TRUE>>=
coords <- getRefCoords(reader,"chr1")
coords
range <- bamRange(reader,coords)
size(range)
@
%
\rcl{bamRange} objects keep a pointer to a current align structure for iteration
purposes. Addidionally there are some summarizing values stored (which are
displayed by \code{show}) which describe the range inside the reference from
which the \code{bamRange} object was read (seqid, qrBegin, qrEnd, complex) and
some statistics (size, qSeqMinLen, qSeqMaxLen). Most of the values are printed
by \rfc{show}:
%
<<>>=
range
getCoords(range)
getSeqLen(range)
getParams(range)
getRefName(range)
@
%
The (0-based) positions of the leftmost and rightmost matching nucleotides in
the align-list are not included by default but can be separately calculated:
%
<<>>=
getAlignRange(range)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{Accessing alignments in bamReader}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\rcl{bamReader} objects keep a list of \rcl{bamAlign} objects. The objects can
sequentially accessed or a data.frame with the alignments data can be retrieved.
Therefore \rcl{bamRange} objects internally keep a pointer to the current alignment.\\
\\
When no current align object is set, the next call to \rfc{getNextAlign} will
set the current to the first alignment in list. When the last alignment in list is
reached, the next call to \rfc{getNextAlign} will return \rcl{NULL}.\\
\\
The \rcl{bamAlign} objects in a \rcl{bamRange} container can be sequentially
accessed with the \rfc{getNextAlign} function:
%
<<>>=
align <- getNextAlign(range)
@
%

After rewinding the \rcl{bamRange} container, the next call to
\rfc{getNextAlign} returns the first stored alignment:
%
<<eval=FALSE>>=
rewind(range)
while(!is.null(align))
{
  # Process align data here
  align <- getNextAlign(range)
}
@
%
A fast way to get tabled alignment information on \rcl{bamRange} container is to
use \rfc{as.data.frame}:
%
<<>>=
rdf <- as.data.frame(range)
@

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{gapList}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\rcl{gapList} objects represent a list of align-gaps. They contain one record
for single each align-gap present in alignment data. Each align-gap can be linked
to a single alignment in the BAM file (via refid and position coordinates).\\
\\
The function \rfc{gapList} takes an open and indexed instance of \rcl{bamReader}
and a set range coordinates (refid,start,stop). The function will scan all
alignments that are overlap with the given range in the opened BAM file for gapped
alignments.\\
\\
For every contained align-gap, the refid and the position of the alignment, the
match length on both sides (\verb|left_cigar_len|, \verb|right_cigar_len|) and
the (1-based) positions of the last nucleotide the left side of the gap
(\verb|left_stop|) and the (1-based) position of the first nucleotide on the
right side of the gap (\verb|right_start|).
%
<<>>=
coords <- getRefCoords(reader, "chr1")
gl <- gapList(reader, coords)
gl
dfr <- as.data.frame(gl)
dfr[1:6, c(1:3, 5:8)]
@
%
The columns 4 and 9 contain the type of the adjacent cigar items
(which should always be 'M') are omitted.\\
\\
The \rfc{size} function returns the number of gaps contained in the object.
The functions \rfc{nAligns} and \rfc{nAlignGaps} return the total number of
alignments and the number of gapped alignments in the scanned range respectively:
%
<<eval=FALSE>>=
size(gl)
nAligns(gl)
nAlignGaps(gl)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{gapSiteList}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\rcl{gapSiteList} objects contain pooled align-gap information. The single gaps
are condensed by refid, left-stop and right-start. So each combination of
coordinates appears only once in the list. The number of alignments in which each
gap has been found is counted into the value \texttt{nAligns}.\\
\\
Two \rcl{gapSitList} objects can be merged to one. The basic coordinates of the
contained gap-sites (refid, lend, rstart) are compared. Gap-sites with no
counterpart are just copied into the new list whereas gap-sites with couterpart
are merged into one record. In this merging process, the core coordinates are
just copied. The following table gives an overview over the calculations which
are done for merging:\\
\begin{center}
\begin{tabular}{lll}
  \hline
    Column name   & Site identificator & Resulting value\\
  \hline
    id            &   & New running index will be created\\
    refid         & + & Copied\\
    lstart        &   & Minimum\\
    lend          & + & Copied\\
    rstart        & + & Copied\\
    rend          &   & Maximum\\
    gaplen        &   & Copied\\
    nAligns       &   & Sum\\
    nProbes       &   & Sum\\
    nlstart       &   & (See text)\\
    \verb|lm_sum| &   & (See text)\\
    lcl           &   & (See text)\\
    mcl           &   & (See text)\\
  \hline
\end{tabular}
\end{center}
%
For \verb|lm_sum|, \texttt{lcl} and \texttt{mcl}, there are specialiced merging
operations.
%
<<>>=
coords <- getRefCoords(reader, "chr1")
sl <- siteList(reader, coords)
size(sl)
nAligns(sl)
nAlignGaps(sl)
sl
df <- as.data.frame(sl)
head(df)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{bamGapList}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\rcl{bamGapList} Objects are designed to contain information about gap-sites for
a complete BAM file (i.e. for all refid's). \rcl{bamGapList}'s can be merged, so
it's possible to cumulate information about gap-sites from a large number of
BAM files (e.g. 50). As the whole collection and merging process is done in C,
the whole process usually runs with a processing rate > 1.000.000 alignments/sec
(on a desktop machine).
%
<<>>=
bsl <- bamGapList(reader)
bsl
size(bsl)
nAligns(bsl)
nAlignGaps(bsl)
summary(bsl)
dfr <- as.data.frame(bsl)
head(dfr)
@

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{readPooledBamGaps}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The two functions
\begin{itemize}
    \item readPooledBamGaps
    \item readPooledBamGapDf
\end{itemize}
provide functionality for extraction and quantification of alignment gap
positions over multiple BAM files.

<<>>=
bam<-system.file("extdata","accepted_hits.bam",package="rbamtools")
rpb<-readPooledBamGaps(bam)
rpdf<-readPooledBamGapDf(bam)
@


The important part of the resulting information is shown below as example.
The boundaries of the covered alignment ranges are denoted
\textit{lstart} and \textit{lend} for the left side and
\textit{rstart} and \textit{rend} for the right side of the alignmnent gap.
Additionally, the number of alignments (nAligns) and the number of samples
(nProbes) which provide support for the alignment gap position are given.
<<echo=TRUE, results=tex>>=
xtable(head(rpdf)[, c(1:6, 8, 9, 13)])
@

Figure \ref{fig:hist_gqs}, although only contains data on 40 gap sites, already
shows the \textit{gqs} distribution at larger samples sizes:
There are many sites with high and low \textit{}gqs values and the
intermediate range is only sparsely filled.

\begin{figure}
    \begin{center}
<<echo=FALSE, fig=TRUE, label=hist_gqs>>=
plot(table(rpdf$gqs), type="h", las=1, bty="n", lwd=2,
        xlab="gqs", ylab="Number of gap sites",
        main="Distribution of gqs values")
@
    \end{center}
    \caption{Distribution of gqs values}
    \label{fig:hist_gqs}
\end{figure}


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\section{Alternative approaches for analyis of splicing events}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Bioconductor alternative to 'readPooledBamGaps'}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
Using Bioconductor alignment numbers on gap sites as well as number of
samples in which gap sites are found can be counted. We show an example
implementation. The function can be evoked by simply providing a vector
of BAM file names (but assumes that associated *.bam.bai BAM index
files are present).

<<eval=FALSE>>=
scanGapSites <- function(bam, yieldSize=1e6, mc.cores=2)
{
    require(Rsamtools)
    require(GenomicAlignments)

    mc.cores <- as.integer(mc.cores)

    # Function will be called by mclapply
    doScanBam <- function(bamFile)
    {
        open(bamFile)

        # Create empty container.
        gPos <- GRanges()

        # Fill container by processing 'yieldSize' reads at a time
        while(
            sum(
                elementLengths(
                    records <- scanBam(
                                bamFile,
                                param=ScanBamParam(
                                    flag=scanBamFlag(isUnmappedQuery=FALSE),
                                    what=scanBamWhat()[c(3, 5, 8)]
                                    )
                                )[[1]]
                    )
            ) > 0
        ){

            nOps <- cigarRangesAlongReferenceSpace(records$cigar,ops="N")
            sel <- elementLengths(nOps) != 0
            gPos <- c(gPos,
                        GRanges(seqnames=rep(records$rname[sel],
                                            elementLengths(nOps)[sel]),
                                            ranges=unlist(shift(nOps[sel],
                                            records$pos[sel]))
                            )
                    )
        }
        close(bamFile)
        # Return all gap positions
        return(gPos)
    }

    cat("[scanGapSites] Processing", length(bam), "Files.\n")


    bamFileList <- BamFileList(bam, yieldSize = yieldSize)
    gList <- mclapply(bamFileList, doScanBam)

    sz <- object.size(gList)
    bm<-Sys.localeconv()[7]
    cat("[scanGapSites] Collected object of size",
            format(as.numeric(object.size(gList)), big.mark=bm),
            "bytes.\n")

    # - - - - - - - - - - - - - - - - - - - - - - - - - #
    # Get all unique positions across all samples
    # - - - - - - - - - - - - - - - - - - - - - - - - - #
    uPos <- unique(Reduce("c", gList))

    # - - - - - - - - - - - - - - - - - - - - - - - - - #
    # Create the count table by
    # transforming the ranges into character strings.
    # - - - - - - - - - - - - - - - - - - - - - - - - - #
    ref <- paste(seqnames(uPos), start(uPos), end(uPos), sep="-")

    # Will be called by mclapply
    doTable <- function(grng, ref)
    {
        tab <- table(paste(seqnames(grng), start(grng), end(grng),sep="-"))
        tab[match(ref,names(tab))]
    }

    count.table <- do.call("cbind",
                    mclapply(gList, doTable, ref, mc.cores=mc.cores))
    rownames(count.table) <- ref

    cat("[scanGapSites] Number of sites:",
            format(nrow(count.table), big.mark=bm),
            ".\n")

    cat("[scanGapSites] Finished.\n")
    return(count.table)
}
@

The \textit{scanGapSites} function offers speed enhancement by parallel
processing and by that may reach similar processing times as the
\textit{readPooledBamGapDf} function.
While multiple chunks of alignment data are loaded into the working memory
demand will be in the range of some Gigabyte (depending on yieldSize).

This function does not provide information on the size of the exonic range
which is covered by the alignment. Also, a quality value similar to the
\textit{gqs} can not be calculated retrospectively.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Direct import of Junction positions from Aligner}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The RNA-seq aligners TopHat and STAR provide information on identified
splice sites in their output files. The Bioconductor \textit{GenomicAlignments}
package provides functions for direct import of this information.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{STAR aligner}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

The STAR aligner ~\cite{pmid23104886} produces a tab separated file where
positions of alignment gaps as well as number of uniquely and multi-mapping
reads and a code for the intron motif is produced (SJ.out.tab file).

The content of these files can directly be imported into R usign the
\textit{readSTARJunctions} method.

% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsubsection{TopHat aligner}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
TopHat ~\cite{pmid23618408} %, pmid23618408, pmid22383036}
produces a
\textit{junctions.bed} file where each junction is
represented in form of two connected BED blocks.
The content of the \textit{junctions.bed} file can be read using the
\textit{readTopHatJunctions} function.

Both tables can be used to obtain candidate locations for splice sites which
also allows fast collection. For further validation of putative splicing
events, additional information may be necessary, e.g. more than just two intronic
nucleotide positions or qualification of the alignment gap postion by the gqs.



% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Miscellaneous functions
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\section{Miscellaneous functions}


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{bamCount and bamCountAll}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The {bamCount} counts alignments and CIGAR-items in alignment ranges defined
by coordinates. The function returns a named integer vector of length 10.\\
\\
The {bamCountAll} counts alignments and CIGAR-items for whole BAM-files
(represented by a \rcl{bamReader}). The function optionally takes a
\code{verbose} argument which controls the textual output during runtime. The
function returns a \rcl{data.frame}. Each line contains counts for one
reference sequence, each column contains data for one CIGAR-OP type. Columns
with total counts, referene sequence id (ID) and reference sequence length (LN)
are added.

<<echo=TRUE,results=tex>>=
coords <- c(0, 0, 14730)
count <- bamCount(reader, coords)
xtable(matrix(count, nrow=1))
@
%
%
<<>>=
count <- bamCountAll(reader, verbose=TRUE)
@

<<echo=FALSE,results=tex>>=
xtable(count, digits=0)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{countNucs}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The {countNucs} counts occurrence of the nucleotides ACGT in
\rcl{bamAlign} and \rcl{bamRange} objects. An integer vector of length 4
is returned. The names give the nucleotide which is counted at each position.
The syntax is identical for \rcl{bamAlign}
<<>>=
align <- bamAlign("HWUSI-0001", "ACCGGGTTTT","Qual/Strng",
                            "4M10N6M", refid=0, position=100)
countNucs(align)
@
and \rcl{bamRange}
<<>>=
reader <- bamReader(bam, idx=TRUE)
coords <- c(0, 0, 14730)
range <- bamRange(reader, coords)
countNucs(range)
@
objects.


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{nucStats}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\paragraph{nucStats for bamReader} The {nucStats} function counts
occurrence of the nucleotides ACGT in whole BAM files via opened
\rcl{bamReader} objects. Any other character values are subsumed in the
value N. The last two columns contain values for GC content and AG/GC ratio.\\
\\
The function returns a \rcl{data.frame} with one row for each reference
sequence which is listed in the BAM-header section.
<<>>=
ncs <- nucStats(reader)
@

<<echo=FALSE, results=tex>>=
xtable(ncs, digits=c(0, 0, 0, 0, 0, 0, 0, 2, 2))
@
%
%
\paragraph{nucStats for BAM file names} The {nucStats} function counts
occurrence of the the nucleotides ACGT for a given list of BAM file names. The
last two columns contain values for GC content and AG/GC ratio. The function
returns a \rcl{data.frame} with one row for each given BAM file name.
<<>>=
ncs <- nucStats(bam)
@

<<echo=FALSE,results=tex>>=
xtable(ncs, digits=c(0, 0, 0, 0, 0, 0, 0, 2, 2))
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{createIdxBatch}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
The {createIdxBatch} is intended to create index files for a multiple BAM-files.
The names of the created BAM-index files can optionally be
added. The standard name for BAM-index files is the name of the BAM file plus
an added suffix ".bai".\\
\\
The third (optional) argument is \rarg{rebuild}. When \rarg{rebuild} is
\code{FALSE} the function will only create not already existing BAM-index files.
When \rarg{rebuild} is \code{TRUE} the function will build BAM-index for all
given BAM-files.\\
\\
Sometimes (especially when BAM-files have been copied), they content may be
corrupt. Rebuilding index files is a way to check the integrity of a BAM-file.
\\
<<eval=FALSE>>=
createIdxBatch(bam)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{readerToFastq, rangeToFastq}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\paragraph{readerToFastq:} The {readerToFastq} and
{rangeToFastq} take (optionally random subsets) of whole BAM-files
(via \rcl{bamReader}) or selected ranges (via \rcl{bamRange}) and copy
alignments to fastq files.\\
\\
For handling of alignments inside whole BAM-files, use the {readerToFastq}
function. Alignments are read from BAM files via {getNextAlign}. For an
opened file, there is a pointer to the last retrieved alignment kept.
Multiple calls to {getNextAlign} will retrieve subsequent alignments.\\
\\
When a logical vector is
given, there will be a call to {getNextAlign} for every entry in the
vector. The function then returns the number of checked alignments.\\
\\
When EOF is reached before the vector is processed, the number of checked alignments
is smaller than the length of the given logical vector. When no logical vector
is given, the function returns the number of written alignments.
<<eval=FALSE>>=
reader <- bamReader(bam)
readerToFastq(reader, "out.fastq")
bamClose(reader)
# Reopen in order to point to first alignment
reader <- bamReader(bam)
index <- sample(1:100, 20)
readerToFastq(reader, "out_subset.fastq", which=index)
@


\paragraph{rangeToFastq:} The function  {rangeToFastq} writes all alignments
in a \rcl{bamRange} object into a compressed fastq file. Optionally, a
logical vector (where length must be equal to size of range) can be given. In
this case only the depicted alignments are copied into the fastq file and the
remaining alings are skipped.
%
<<eval=FALSE>>=
reader <- bamReader(bam, idx=TRUE)
coords <- as.integer(c(0,0,249250621))
range <- bamRange(reader,coords)
rangeToFastq(range,"rg.fq.gz")
index <- sample(1:size(range),100)
rangeToFastq(range,"rg_subset.fq.gz",which=index)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Functions for reading and displaying Phred quality scores}
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

Phred quality scores Q are defindes as $Q=-10 log_{10} P$ where P is the base
calling error probability.

\paragraph{getQualDf} takes a \rcl{bamReader} and returns a data.frame. The
\rcl{data.frame} has 94 rows which represent values from 0 to 93
(~\cite{fastq}). The number of columns equals the maximum sequence length in
the given \rcl{bamRange}.\\

<<>>=
qdf <- getQualDf(range)
qdf[32:38,1:10]
qdr <- getQualDf(range,prob=TRUE)
qrr <- round(qdr,2)
qrr[32:38,1:10]
@

\paragraph{getQualQuantiles} takes a \rcl{bamReader} and a vector of quantiles
(must be between 0 and 1) and returns a data.frame. The data.frame contains one
row for each quantile and also as many columns as the maximum sequence length.\\

<<>>=
qt <- getQualQuantiles(range,c(0.25,0.5,0.75))
qt[,1:10]
@

\paragraph{plotQualQuant} takes a \rcl{bamReader} and plots the 0.1, 0.25, 0.5,
0.75 and 0.9 quantiles for all occurring sequence positions.

<<eval=TRUE,fig=TRUE>>=
plotQualQuant(range)
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Covers alignDepth S4 class
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Functions for calculation and displaying alignment-depth}

Alignment depth means quantification of present matches for each nucleotide position
in a given range.

\paragraph{The alignDepth member function} calculates alignment depth for a given
bamRange object. From the \rcl{bamRange} object, the range is extracted and for
each nucleotide position whithin this range the numbers of alignment matches are
calculated. When \rcl{alignDepth} is called wich \robj{gap=TRUE}, the function
counts alignments solely for gap-adjacent match regions (cigar-op's).\\

Whe extract a \rcl{bamRange} for the WASH7:

<<eval=TRUE>>=
# WASH7P coordinates
xlim <- c(10000, 30000)
coords <- c(0,xlim[1], xlim[2])
range <- bamRange(reader, coords)
bamClose(reader)
ad <- alignDepth(range)
ad
getParams(ad)
# Identifier
gene <- "WASH7P"
ensg_id <- "ENSG00000227232"
enst_id <- "ENST00000538476"
# Get exon positions
start <- c(14411, 15000, 15796, 15904, 16607, 16748, 16858, 17233,
                    17602, 17915, 18268, 24737, 29534)
end <-   c(14502, 15038, 15901, 15947, 16745, 16765, 17055, 17364,
                    17742, 18061, 18366, 24891, 29806)
@

<<eval=TRUE,fig=TRUE>>=
plotAlignDepth(ad, lwd = 2, xlim = xlim,
            main = paste("Align depth for gene",gene),
            ylab = "Align depth", start = start,
            end = end, strand = "-",
            transcript = paste("Chromosome 1",
                "\tGene ENSG00000227232", ensg_id,
                "\tTranscript ",enst_id
))
@


% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Covers rangeSegCount class
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
\subsection{Functions for counting alignmnents in genomic segments}

The \rcl{rangeSegCount} class counts alignment numbers in specified
genomic segments. The \rfc{rangeSegCount} function takes three arguements:
\begin{itemize}
    \item reader: An opened instance of bamReader which must contain an
        initialized bam index.
    %
    \item coords: A numeric vector of length 3 which contains
        (seqid, start, end) as specified for bamRange object.
        As for bamRange's, this defines the genetic region from which
        alignments are read from a BAM file.
    %
    \item segments: A numeric vector of arbitrary length which should
        divide the range defined by \rfc{coords} into segments. In the
        segments vector, two adjacent values define a right open interval
        in which the genomic alignments are counted.
\end{itemize}

The function only considers the alignment start postions. This reflects the
difference to the alignment depth construction where aligned positions are
counted separately for each nucleotide.

<<>>=
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
# B) Count range segment
# - - - - - - - - - - - - - - - - - - - - - - - - - - - - #
reader <- bamReader(bam, idx=TRUE)
coords <- c(0, 0, 2e4)
segments <- seq(14000, 20000, 20)
segcount<-rangeSegCount(reader, coords, segments)
segcount
dfr<-as.data.frame(segcount)
sum(dfr$count)

@

<<eval=TRUE, fig=TRUE>>=
plot(count~position, dfr, type="l",
        las=1, bty="n", lwd=1.5, col="dodgerblue2",
        xlab="Position on Chromosome 1",
        ylab="Alignment count",
        main="Number of alignments in genomic segments of 20 nucleotides size")
@


%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%% Experimental data from multiple samples
%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%
\section{Experimental data from multiple samples}

Sequencing is usually done in experimental settings where multiple
samples are distibuted over several groups.
Multiple samples in an experimental design can be handled using
the \rcl{sampleBamFiles} class.\\
\\
The central idea is to be able to plot alignment depth for
genetic regions from a whole experiment, so that different
alignement depths over genes or exons can be visualized.

<<>>=
bs <- sampleBamFiles(6)
bamFiles(bs) <- paste("bam",1:6, ".bam", sep="")
sampleLabels(bs) <- paste(rep(c("wt", "trt"), each=3), 1:6, sep="")
sampleGroups(bs) <- rep(c("wildtype", "treated"), each=3)
bs
@

Sample labels are intended as short identifiers used for example in
plots.\\
\\
Using this type of instantiation automatically generates BAM index file
names of type "*.bai":
<<>>=
bamIdxFiles(bs)
@

This default can be overwritten by assignment

<<>>=
bamIdxFiles(bs)<- paste("index", 1:6, ".bam.bai", sep="")
@

\subsection{Checking BAM files}

The check routine verifies crucial prerequisites for \rcl{sampleBamFiles} usage:

\begin{itemize}
    \item Existing BAM and index files
    \item Congruent Reference sequence (chromosome) names
\end{itemize}

\subsection{Counting BAM files}

Visualization of alignment depth data on single genes requires
normalization of alignment count values to a common sequencing depth.
The total alignment numbers can easily be obtained using
the \rfc{bamCountAll} function.

<<eval=FALSE>>=
nAligns(bs) <- bamCountAll(bs)
@


\subsection{geneAlignDepth}

Align depth data from BAM files is read and stored in
\rcl{geneAlignDepth} objects. They can be created from
\rcl{sampleBamFiles} and \rcl{geneModel} (located in refGenome)
objects.\\
\\
The construction process is divided in several steps as shown below.

\subsubsection{Construction of bamFiles object}

<<>>=
bam<-system.file("extdata", "accepted_hits.bam", package="rbamtools")
bs <-sampleBamFiles(1)
bamFiles(bs) <- bam
sampleLabels(bs) <- "s1"
sampleGroups(bs) <- "g1"
checkBamFiles(bs)
nAligns(bs) <- bamCountAll(bs)
bs
@

\subsubsection{Construction of geneModel}

A \rcl{geneModel} object can be constructed using reference
genome as implemented in package refGenome. A \rcl{geneModel}
object contains data on single genes for example
\begin{itemize}
    \item Gene ID (e.g. Ensembl identifier)
    \item Gene name (e.g. HGNC symbol)
    \item Genetic coordinates (chromosome, start, end)
    \item Transcripts
    \item Positions of exons and CDS
\end{itemize}

<<>>=
## - - - - - - - - - - - - - - - - - - - - - - ##
# Construct geneModel object
ucfile<-system.file("extdata", "hs.ucsc.small.RData", package="refGenome")
uc<-loadGenome(ucfile)
gt <- getGeneTable(uc)
gene_id <- as.character(gt$gene_id[1])
gm <- geneModel(uc, gene_id)
@

\subsubsection{Read alignment depth for a whole gene}

Sequencing experiments usually include more than one sample.
\rcl{rbamtools} provides a framework for analysis of
alingment depth for a single gene on multiple BAM files.\\
%
Aside from counting alignment depth, the challenge posed by this task
arises from the large proportion of introns in the range of a gene
making a visual inspection of raw data impractical.\\
%
What needs to be done is to identify as much intronic region as possible
while restricting the search algorithm to traceable criterions.\\
%
Relying on annotation data quickly emerges as unfeasible because different
transcripts possibly imply conflictive definitions of exonic regions
e.g. when exons are skipped.\\
%
Additionally, a groupwise measures for alignemnt depth are required for
interpretation. \rcl{rbamtools} performes groupwies loess derived
estimates for global align-depth normalized data.\\
%
A central idea is, that cutting out intronic regions can be divided
into multiple consecutive steps allowing the user to direct the
removal of regions to a sensible level.\\
%
The algorithms implemented in \rcl{rbamtools} provide multiple consecutive
processing steps:
\begin{itemize}
    \item \rcl{geneAlignDepth}
    \item \rcl{exonAlignDepth}
    \item \rcl{exonLoessModel}
    \item \rcl{cutFlatAlignDepth}
\end{itemize}

\paragraph{geneAlignDepth} A \rcl{geneAlignDepth} object
can be crated from a \rcl{sampleBamFiles} and a \rcl{geneModel} object.\\
%
For usage in \rcl{geneAlignDepth} the \rcl{geneModel} can be
instantiated using 'interior=FALSE' option which omits
extraction of transcript data.
During the creation process, the align depth data is extracted from
the BAM files. The calculated aling depth can be visualized using
a generic plot function.

<<fig=TRUE>>=
gm <- geneModel(uc, gene_id, interior=FALSE)
gad <- geneAlignDepth(bs, gm)
plot(gad, col="gray50")
@

The \rfc{exonAlignDepth} function cuts out intronic regions from
\rcl{geneAlignDepth} data. The intronic regions are identified using
exon-intron boundaries.
Exon-intron boundaries are localized by two criterions:
\begin{enumerate}
    \item Begin or end of an alignment gap
    \item A high ratio of alignment depth between adjacent exonic and intronic
        nucleotide.
\end{enumerate}
Using the identified exon-intron, the intronic regions are cut out of the
using a \rfc{segmentize} function defined in \rcl{rbamtools}.
As the implementation of \rfc{segmentize} uses a logical vector for
removal, the removed positions are the union of the defined
intronic segments which is sensible when intronic regions overlap.

<<fig=TRUE>>=
ead <- exonAlignDepth(gad, ratioLim=5, infVal=1000)
plot(ead)
@


<<fig=TRUE>>=
elm <- exonLoessModel(ead)
plot(elm)
@

<<>>=
celm <- cutFlatAlignDepth(elm, ratio=0.1)
plot(celm)
@

<<>>=
junctionSites(ead)
groupRatio(celm, lim=1.2, cut=0, order=1)
@

%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%% Misalign errors
%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%
\section{Misalign errors}
%
\subsection{Cause of misalign errors}
In the samtools C library, alignment data is contained in \texttt{bam1\_t}
structures.
Cigar data is stored in the \texttt{data} segment of \texttt{bam1\_t}
structures which has type \texttt{unsigned char *} (1 byte).
The data data segments usually are accessed
using pointers to \texttt{unsigned char} as shown below.
%
\begin{verbatim}
bam1_t *align = bam_init1();
unsigned char * data = align->data;
\end{verbatim}
%
Cigar data has type \texttt{unsigned int} (4 bytes) which differs
from the type size of \texttt{unsigned char}.
%
In order to retrieve cigar data from an alignment structure a type
change has to be performed. On pointers, these type changes are called
pointer casts.
%
The accessor for cigar data \texttt{bam1\_cigar} performs
an implicit pointer cast in the following statment:
%
\begin{verbatim}
unsigned int *cigar=align->data + align->core.l_qname;
\end{verbatim}
%
Because there are usually multiple cigar items for each alignment, a
cigar structure actually contains more than one value. Therefore
the value pointed to is an array rather than a single value.
%
Values inside a C array are accessed via the index operator '[x]'
(similar to R). The first value inside a cigar array is accessed using
\begin{verbatim}
unsigned int c = cigar[0];
\end{verbatim}
(unlike R where the first element has index 1).
%
In other words, the index operator k[x] refers to a memory location
x-units away from the base address k. The unit size is the memory
demand of one element of the stored type.
%
By putting the example together we see that after
\begin{verbatim}
bam1_t *align = bam_init1();
unsigned char * data = align->data + align->core.l_qname;
unsigned int *cigar = data;
\end{verbatim}
the two expressions
\begin{verbatim}
data[4];
cigar[4];
\end{verbatim}
point to different memory locations due to the implicit conversion during
the assignment from data to cigar.

For this ambiguity, there is no general rule defined in the C language
definition, a so called \textbf{undefined behaviour} (UB) situation.
The result may depend on implementation and may vary between different
operating systems for example between Debian Linux and SPARC.

The accountable structures currently are present in all C implementations of
samtools, for example in the current (Bioconductor) Rsamtools
version\footnote{http://www.bioconductor.org/packages/release/bioc/html/Rsamtools.html}
and in the current htslib version (0.2.0-rc8) on
GitHub\footnote{https://github.com/samtools/samtools}.


%
%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%% Misalign causing code in htslib:
%% ~/htslib/htslib-0.2.0-rc8/htslib/sam.h
%% line 161: #define bam_get_cigar(b) ((uint32_t*)((b)->data + (b)->core.l_qname))
%% Misalign causing code in Rsamtools:
%% Rsamtools/src/samtools/bam.h:
%% line 225: #define bam1_cigar(b) ((uint32_t*)((b)->data + (b)->core.l_qname))
%% A good explanation for effects of unaligned memory access:
%% http://askldjd.wordpress.com/2009/12/07/memory-alignment-problems/
%% + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + %%
%%
%
\subsection{Correction of misalign errors}
%
In rbamtools, we have corrected these misalign errors by a new \texttt{cigar}
data member into \texttt{bam1\_t} structures and maintaining a
copy of cigar data there.
%
The implementation of this workaround requires changes in five functions:
The creator (\texttt{bam\_init1}) and destructor (\texttt{bam\_destroy1}),
the functions for copying  (\texttt{bam\_copy1}) and duplication
(\texttt{bam\_dup1}) and the accessor for cigar data (\texttt{bam1\_cigar)}.
%
Additionally, the samtools library has to be corrected at places where the own
creator and destructor routines had not been deployed (e.g. in the
\texttt{bam\_sort\_core\_ext}  function in bam\_sort.c and in the
\texttt{bam\_index\_core} function in bam\_index.c).
%
This workaround is associated with a slight speed decrease of 6.7 \% but it
does not change file access and the programming interface.
It can be introduced into any current samtools version.
%
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %



% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %
% Bibliography
% - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - %

\bibliographystyle{plain}
\bibliography{rbamtools}{}
\end{document}
