<CRANTaskView>

  <name>ReproducibleResearch</name>
  <topic>Reproducible Research</topic>
  <maintainer email="max.kuhn@pfizer.com">Max Kuhn</maintainer>
  <version>2015-12-03</version>
  
  <info>
    <p>
      The goal of reproducible research is to tie specific instructions to data analysis and experimental data so that scholarship can be recreated, better understood and verified. Packages in R for this purpose can be split into groups for: literate programming, package reproducibility, code/data formatting tools, format convertors, and object caching. 
    </p>
    <h1>Literate Programming</h1>
    <p>
      The primary way that R facilitates reproducible research is using a document that is a combination of content and data analysis code. The <code>Sweave</code> function (in the base R utils package) and the <pkg>knitr</pkg> package can be used to blend the subject matter and R code so that a single document defines the content and the analysis. The <pkg>brew</pkg>  and <pkg>R.rsp</pkg> packages contain alternative approaches to embedding R code into various markups.
    </p>
    <p>
      The resources for literate programming are best organized by the document type/markup language:
    </p> 
    <h2>
      LaTeX
    </h2>
      <p>
        Both <code>Sweave</code> and <pkg>knitr</pkg> can process LaTeX files. <pkg>lazyWeave</pkg> can create LaTeX documents from scratch. 
      </p>
      <p>
        Object Conversion Functions:
      </p>
        <ul>
          <li>
            <i>summary tables/statistics</i>: <pkg>Hmisc</pkg>, <pkg>NMOF</pkg>, <pkg>papeR</pkg>, <pkg>quantreg</pkg>, <pkg>rapport</pkg>, <pkg>reporttools</pkg>, <pkg>sparktex</pkg>, <pkg>tables</pkg>,<pkg>xtable</pkg>, <pkg>ztable</pkg>
          </li>    
          <li>
            <i>tables/cross-tabulations</i>: <pkg>compareGroups</pkg>, <pkg>Hmisc</pkg>, <pkg>lazyWeave</pkg>, <pkg>knitLatex</pkg>, <pkg>knitr</pkg>, <pkg>reporttools</pkg>, <pkg>ztable</pkg>
          </li>      
          <li>
            <i>graphics</i>: <pkg>animation</pkg>, <pkg>Hmisc</pkg>, <code>grDevices:::pictex</code>, <pkg>sparktex</pkg>, <pkg>tikzDevice</pkg> 
          </li>    
          <li>
            <i>statistical models/methods</i>: <pkg>apsrtable</pkg>, <pkg>memisc</pkg>, <pkg>quantreg</pkg>,  <pkg>r2lh</pkg>, <pkg>rms</pkg>, <pkg>stargazer</pkg>, <pkg>suRtex</pkg>, <pkg>TeachingSampling</pkg>, <pkg>texreg</pkg>, <pkg>xtable</pkg>, <pkg>ztable</pkg>
          </li> 
          <li>
            <i>bibtex</i>: <pkg>bibtex</pkg> and <pkg>RefManageR</pkg>
          </li> 
          <li>
            <i>others</i>: <pkg>latex2exp</pkg> converts LaTeX equations to <tt>plotmath</tt> expressions. 
          </li>                                                   
        </ul>  
      <p>
        Miscellaneous Tools
      </p>
        <ul>
          <li>
            <pkg>Hmisc</pkg> contains a function to correctly escape special characters. <pkg>resumer</pkg> creates resumes. Standardized exams can be created using the <pkg>exams</pkg> package.
          </li>                                                   
        </ul> 
   <h2>
      HTML
    </h2>
      <p>
        The <pkg>knitr</pkg> package can process HTML files directly.  <code>Sweave</code> can also work with HTML by way of the <pkg>R2HTML</pkg> package. <pkg>Kmisc</pkg>  and <pkg>lazyWeave</pkg> can create HTML format documents from scratch.  
      </p>
      <p>
        Object Conversion Functions:
      </p>
        <ul>
          <li>
            <i>summary tables/statistics</i>: <pkg>stargazer</pkg>
          </li>    
          <li>
            <i>tables/cross-tabulations</i>: <pkg>compareGroups</pkg>, <pkg>DT</pkg>,  <pkg>formattable</pkg>, <pkg>htmlTable</pkg>, <pkg>HTMLUtils</pkg>, <pkg>hwriter</pkg>, <pkg>Kmisc</pkg>, <pkg>knitr</pkg>, <pkg>lazyWeave</pkg>, <pkg>SortableHTMLTables</pkg>, <pkg>texreg</pkg>, <pkg>ztable</pkg>
          </li>      
          <li>
            <i>statistical models/methods</i>: <pkg>r2lh</pkg>, <pkg>rapport</pkg>, <pkg>stargazer</pkg>, <pkg>xtable</pkg>
          </li> 
          <li>
            <i>others</i>: <pkg>knitcitations</pkg>, <pkg>RefManageR</pkg>
          </li>                                                   
        </ul>  
      <p>
        Miscellaneous Tools: <pkg>htmltools</pkg> has various tools for working with HTML. <pkg>tufterhandout</pkg> for creating Tufte-style handouts
      </p>

   <h2>
      Markdown
    </h2>
      <p>
        The <pkg>knitr</pkg> package can process markdown files without assistance. The packages  <pkg>markdown</pkg> and <pkg>rmarkdown</pkg> have general tools for working with documents in this format. <pkg>Kmisc</pkg> and <pkg>lazyWeave</pkg> can create markdown format documents from scratch. Also, the <pkg>ascii</pkg> package can write R objects to the <a href="http://www.methods.co.nz/asciidoc/">AsciiDoc</a> format. 
      </p>
      <p>
        Object Conversion Functions:
      </p>
        <ul>
          <li>
            <i>summary tables/statistics</i>: <pkg>papeR</pkg>
          </li>    
          <li>
            <i>tables/cross-tabulations</i>:  <pkg>DT</pkg>, <pkg>formattable</pkg>, <pkg>htmlTable</pkg>, <pkg>knitr</pkg>, <pkg>Kmisc</pkg>, <pkg>lazyWeave</pkg>, <pkg>papeR</pkg>
          </li>        
          <li>
            <i>statistical models/methods</i>: <pkg>pander</pkg>, <pkg>papeR</pkg>, <pkg>rapport</pkg>, <pkg>texreg</pkg>
          </li> 
          <li>
            <i>others</i>: <pkg>RefManageR</pkg>
          </li>                                                   
        </ul>  
      <p>
        Miscellaneous Tools: <pkg>tufterhandout</pkg> for creating Tufte-style handouts. <pkg>kfigr</pkg> allows for figure indexing in markdown documents. 
      </p>

   <h2>
      OpenDocument Format (ODF)
    </h2>
      <p>
        The <pkg>odfWeave</pkg> package can process ODF files.  
      </p>
      <p>
        Object Conversion Functions:
      </p>
        <ul>
          <li>
            <i>tables/cross-tabulations</i>: <pkg>odfWeave</pkg>
          </li>        
          <li>
            <i>statistical models/methods</i>: <pkg>odfWeave.survey</pkg>, <pkg>rapport</pkg>
          </li>                                                  
        </ul>  

   <h2>
    Microsoft Formats
   </h2>
   <p>
    The <pkg>ReporteRs</pkg> (formerly R2DOCX) package can create <tt>docx</tt> and <tt>pptx</tt> files.  <pkg>R2wd</pkg> (windows only) can also create Word documents from scratch and <pkg>R2PPT</pkg> (also windows only) can create PowerPoint slides. The <pkg>rtf</pkg> package does the same for Rich Text Format documents. 
   </p>
   <p>
    Object Conversion Functions:
   </p>
   <ul>   
    <li>
      <i>tables/cross-tabulations</i>: <pkg>apaStyle</pkg>
    </li>
   </ul>  

   <h1>
     Miscellaneous Tools
   </h1>     

   <h2>Package Reproducibility</h2>    
   <p> 
     R also has tools for ensuring that specific packages versions can be required for analyses. <pkg>checkpoint</pkg>, <pkg>rbundler</pkg> and <pkg>packrat</pkg> install packages required for a project to a local archive as they existed at a specified point in time. This allows specific package versions to be maintained over time and different users. The <pkg>miniCRAN</pkg> package facilitates the creation of local CRAN-like repositories. 
   </p>

   <h2>Formatting Tools</h2> 
   <p>
    <pkg>formatR</pkg>, <pkg>highlight</pkg>, <pkg>highr</pkg>, and <pkg>SweaveListingUtils</pkg> can be used to color and/or format R code.                                                 
   </p>  
   <p>
    Packages <pkg>humanFormat</pkg>, <pkg>lubridate</pkg>, <pkg>prettyunits</pkg>, and <pkg>rprintf</pkg> have functions to better format data. 
   </p>

   <h2>Format Convertors </h2> 
   <p>
    <pkg>pander</pkg> can be used for rendering R objects into <a href="http://pandoc.org/">Pandoc's</a> markdown. <pkg>knitr</pkg> has the function <tt>pandoc</tt> that can call an installed version of <a href="http://pandoc.org/">Pandoc</a> to convert documents between formats such as Markdown, HTML, LaTeX, PDF and Word. <pkg>tth</pkg> facilitates TeX to HTML/MathML conversions. LaTeX to Rich Text Format converions can be done using the <pkg>connect3</pkg> package. 
   </p>   

   <h2>Object Caching Packages</h2> 
   <p>
    When using <code>Sweave</code> and <pkg>knitr</pkg> it can be advantageous to <i>cache</i> the results of time consuming code chunks if the document will be re-processed (i.e. during debugging). <pkg>knitr</pkg> facilitates object caching and the Bioconductor package <bioc>weaver</bioc> can be used with <code>Sweave</code>.
   </p>   
   <p>
   Non-literate programming packages to facilitating caching/archiving are <pkg>R.cache</pkg> and <pkg>archivist</pkg>.
   </p>
  </info>

  <packagelist>
    <pkg>animation</pkg> 
    <pkg>apaStyle</pkg>  
    <pkg>apsrtable</pkg>
    <pkg>archivist</pkg>    
    <pkg>ascii</pkg>
    <pkg>bibtex</pkg>
    <pkg>brew</pkg>
    <pkg>checkpoint</pkg>   
    <pkg>compareGroups</pkg>
    <pkg>connect3</pkg>
    <pkg>DT</pkg>
    <pkg>exams</pkg>
    <pkg>formatR</pkg>
    <pkg>formattable</pkg>   
    <pkg>highlight</pkg>    
    <pkg>highr</pkg>      
    <pkg>htmlTable</pkg>
    <pkg>htmltools</pkg> 
    <pkg>HTMLUtils</pkg>
    <pkg>humanFormat</pkg>
    <pkg>hwriter</pkg>
    <pkg priority="core">Hmisc</pkg>
    <pkg>latex2exp</pkg>
    <pkg>lazyWeave</pkg>   
    <pkg>lubridate</pkg>    
    <pkg>kfigr</pkg>
    <pkg>Kmisc</pkg>
    <pkg>knitcitations</pkg>
    <pkg>knitLatex</pkg>
    <pkg priority="core">knitr</pkg>
    <pkg>markdown</pkg>
    <pkg>memisc</pkg>
    <pkg>miniCRAN</pkg>
    <pkg>NMOF</pkg>
    <pkg>odfWeave</pkg>
    <pkg>odfWeave.survey</pkg>
    <pkg>packrat</pkg>       
    <pkg>pander</pkg>
    <pkg>papeR</pkg>
    <pkg>prettyunits</pkg>
    <pkg>quantreg</pkg>
    <pkg>R.cache</pkg>
    <pkg>R.rsp</pkg>
    <pkg priority="core">R2HTML</pkg>
    <pkg>r2lh</pkg>
    <pkg>R2PPT</pkg>
    <pkg>R2wd</pkg>
    <pkg>rapport</pkg>
    <pkg>rbundler</pkg>
    <pkg>RefManageR</pkg>
    <pkg>reporttools</pkg>    
    <pkg>ReporteRs</pkg>
    <pkg>resumer</pkg>
    <pkg>rmarkdown</pkg>
    <pkg priority="core">rms</pkg>
    <pkg>rprintf</pkg>
    <pkg>rtf</pkg>
    <pkg>SortableHTMLTables</pkg>
    <pkg>sparktex</pkg>
    <pkg>stargazer</pkg>
    <pkg>suRtex</pkg>
    <pkg>SweaveListingUtils</pkg>
    <pkg>tables</pkg>
    <pkg>TeachingSampling</pkg>
    <pkg>texreg</pkg>
    <pkg>tikzDevice</pkg>
    <pkg>tth</pkg>
    <pkg>tufterhandout</pkg>
    <pkg priority="core">xtable</pkg>
    <pkg>ztable</pkg>
  </packagelist>

  <links>
    <a href="http://www.statistik.uni-muenchen.de/~leisch/Sweave/">The Sweave Homepage</a>
    <a href="http://yihui.name/knitr/">knitr: Elegant, flexible and fast dynamic report generation with R</a>
    <bioc>weaver</bioc>
    <a href="http://en.wikipedia.org/wiki/Literate_Programming">Wikipedia: Literate Programming</a>
    <a href="http://biostat.mc.vanderbilt.edu/wiki/Main/StatReport">Harrell: Site for Reproducible Research </a>
    <a href="http://www.econ.uiuc.edu/~roger/research/repro/">Koenker, Zeileis: On Reproducible Econometric Research</a>
    <a href="http://biostatistics.oxfordjournals.org/cgi/reprint/10/3/405">Peng: Reproducible Research and Biostatistics</a>
    <a href="http://www.bepress.com/uwbiostat/paper194/">Rossini, Leisch: Literate Statistical Practice</a>
    <a href="http://dx.doi.org/10.1214/09-AOAS291">Baggerly, Coombes: Deriving Chemosensitivity from Cell Lines: Forensic Bioinformatics and Reproducible Research in High-Throughput Biology</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2002-3.pdf">Leisch: Sweave, Part I: Mixing R and LaTeX</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2003-2.pdf">Leisch: Sweave, Part II: Package Vignettes</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2005-1.pdf">Betebenner: Using Control Structures with Sweave</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2006-2.pdf">Garbade, Burgard: Using R/Sweave in Everyday Clinical Practice</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2006-4.pdf">Kuhn: Sweave and the Open Document Format - The odfWeave Package</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2008-1.pdf">Gorjanc: Using Sweave with LyX</a>
    <a href="http://www.R-project.org/doc/Rnews/Rnews_2003-3.pdf">Lecoutre: The R2HTML Package</a>
  </links>



</CRANTaskView>
