<?xml version="1.0" encoding="UTF-8"?>
<CRANTaskView>

<name>ExperimentalDesign</name>
<topic>Design of Experiments (DoE) &amp; Analysis of Experimental Data</topic>
<maintainer email="groemping@bht-berlin.de">Ulrike Groemping</maintainer>
<version>2016-08-18</version>

<info>
  <p>This task view collects information on R packages for experimental design
  and analysis of data from experiments. With a strong increase in the number of 
  relevant packages, packages that focus on analysis only and do not make relevant 
  contributions for design creation are no longer added to this task view. 
   Please feel free to
  suggest enhancements, and please send information on new packages or major 
  package updates if you think they belong here. Contact details are given on my
  <a href="http://prof.beuth-hochschule.de/groemping/DoE">Web page</a>.</p>
  
  <p>Experimental design is applied in many areas, and methods have been tailored 
  to the needs of various fields. This task view starts out with a section on 
  the most general packages, continues with specific sections on agricultural and 
  industrial experimentation, computer experiments, and experimentation in the
  clinical trials contexts, and closes with a section on various special 
  experimental design packages that have been developed for other specific purposes.
  Of course, the division into fields is not always clear-cut, and some packages from 
  the more specialized sections can also be applied in general contexts. <br/>
  You may also notice that my own experience is mainly from industrial experimentation 
  (in a broad sense), which may explain a somewhat biased view on things.</p>

  <h3>Experimental designs for agricultural and plant breeding experiments</h3>
  <p>Package <pkg>agricolae</pkg> is by far the most-used package from this task view (status: August 2016). 
  It offers extensive functionality on experimental design 
  especially for agricultural and plant breeding experiments, which can also be useful 
  for other purposes. It supports <i>planning</i> of lattice designs, factorial designs, 
  randomized complete block designs, completely randomized designs, 
  (Graeco-)Latin square designs, balanced incomplete block designs and alpha designs. 
  There are also various <i>analysis</i> facilities for experimental data, e.g. treatment 
  comparison procedures and several non-parametric tests, but also some quite specialized 
  possibilities for specific types of experiments. Package <pkg>desplot</pkg> is made 
  for plotting the layout of agricultural experiments. Package <pkg>agridat</pkg> 
  offers a large repository of useful agricultural data sets.</p>

  <h3>Experimental designs for general purposes</h3>
  <p>There are a few packages for creating and analyzing experimental designs
  for general purposes: First of all, the standard (generalized) linear model
  functions in the base package stats are of course very important for analyzing
  data from designed experiments (especially functions <code>lm()</code>, <code>aov()</code>
  and the methods and functions for the resulting linear model objects). These are
  concisely explained in Kuhnert and Venables (2005, p. 109 ff.); Vikneswaran (2005)
  points out specific usages for experimental design (using function <code>contrasts()</code>,
  multiple comparison functions and some convenience functions like <code>model.tables()</code>,
  <code>replications()</code> and <code>plot.design()</code>). 
  Lawson (2014) is a good introductory textbook on experimental design in R, which 
  gives many example applications. 
  Lalanne (2012)   provides an R companion to the well-known book by Montgomery (2005); 
  he so far covers approximately the first 
  ten chapters; he does not include R's design generation facilities, but mainly 
  discusses the analysis of existing designs. 
  Package <pkg>GAD</pkg> handles 
  general balanced analysis of variance models with fixed and/or random effects 
  and also nested effects (the latter can only be random); they quote Underwood 1997 for this work.
  The package is quite valuable, as many 
  users have difficulties with using the R packages for handling random or mixed effects. 
  Package <pkg>granova</pkg> 
  offers some interesting non-standard graphical representations for results of simply-structured 
  experiments (one-way and two-way layouts, paired data), package <pkg>ez</pkg> aims at supporting 
  intuitive analysis and visualization of factorial experiments based on package "ggplot2".
  </p> 

  <ul>
  <li>Package <pkg>AlgDesign</pkg> creates full
  factorial designs with or without additional quantitative variables, creates mixture
  designs (i.e., designs where the levels of factors sum to 1=100%; lattice designs are created only) and creates
  D-, A-, or I-optimal designs exactly or approximately, possibly with blocking, using the Federov (1972) algorithm. 
  Package <pkg>OptimalDesign</pkg> likewise calculates unblocked D-, A-, or I-optimal designs 
  (they use "IV-optimal" instead of "I-optimal") exactly or approximately, 
  treating quantitative variables only, including mixture designs; 
  this package uses different algorithms (e.g. Atkinson, Donev and Tobias 2007, Harman and Filova 2014), 
  some of which rely on the availability of the gurobi software 
  (<a href="http://www.gurobi.com/">http://www.gurobi.com/</a>, free for academics and academic institutions) 
  and its accompanying R package "gurobi" (not on CRAN).</li> 
  <li>Package <pkg>ICAOD</pkg> implements the 
  "Imperialist Competitive Algorithm for Optimal Designs" for nonlinear models 
  according to Masoudi, Holling and Wong (2016). 
  Package <pkg>LDOD</pkg> implements locally D-optimal designs 
  for some nonlinear and generalized linear models, package <pkg>designGLMM</pkg> locally 
  optimal designs for completely randomized or blocked Poisson models, 
  and package <pkg>PopED</pkg> optimal designs for nonlinear mixed effect models.</li> 
  <li>There are various further packages that deal with optimal designs of different types: 
  package <pkg>oapackage</pkg> 
  allows to generate (not necessarily orthogonal) 
  optimal fractional factorial 2-level designs with the possibility to prioritize 
  main effects higher than interactions. Package <pkg>rodd</pkg> provides T-optimal designs, 
  also called optimal discriminating designs (Dette, Melas and Shpilev 2013,
  Dette, Melas  and Guchenko 2014), Package <pkg>acebayes</pkg>
  calculates optimal Bayesian designs using an approcimate coordinate exchange algorithm, 
  package <pkg>OBsMD</pkg> provides "Objective Bayesian Model Discrimination in Follow-Up Designs" 
  according to Consonni and Deldossi (2015). Further optimal design packages for 
  very specific purposes are listed at the end of this view.
  </li> 
  <li>Package <pkg>conf.design</pkg> allows 
  to create a design with certain interaction effects confounded with blocks (function
  <code>conf.design()</code>) and allows to combine existing designs in several ways
  (e.g., useful for Taguchi's inner and outer array designs in industrial experimentation). </li>
  <li>Package <pkg>planor</pkg> allows 
  to generate regular fractional factorial designs with fixed and mixed levels 
  and quite flexible randomization structures. The packages flexibility 
  comes at the price of a certain complexity and - for larger designs - high computing time.</li>
  <li>Package <pkg>ibd</pkg> creates and analyses incomplete block designs. Packages 
  <pkg>PGM2</pkg>, <pkg>RPPairwiseDesign</pkg> and <pkg>CombinS</pkg> all produce designs 
  related to (resolvable) (partially) balanced incomplete block designs.</li>
  <li>Package <pkg>crossdes</pkg> creates and analyses cross-over designs of various types (including
  latin squares, mutually orthogonal latin squares and Youden squares) that can for example
  be used in sensometrics. Package <pkg>Crossover</pkg> also provides crossover designs; 
  it offers designs from the literature and algorithmic designs, makes use of the 
  functionality in <pkg>crossdes</pkg> and in addition provides a GUI.</li>
  <li>Package <pkg>DoE.base</pkg> provides full factorial designs with or without blocking 
  (function <code>fac.design</code>) and orthogonal arrays (function <code>oa.design</code>) 
  for main effects experiments 
  (those listed by Kuhfeld 2009 up to 144 runs, plus a few additional ones). 
  There is also some functionality for assessing the quality of orthogonal arrays, 
  related to Groemping and Xu (2014), and some analysis functionality with half-normal effects plots in 
  quite general form (Groemping 2015).<br/>
  Package <pkg>DoE.base</pkg> also forms the basis of a suite of related packages (cf. Groemping 2009). 
  Together with <pkg>FrF2</pkg> (cf. below) and <pkg>DoE.wrapper</pkg>, it provides the work horse 
  of the GUI package <pkg>RcmdrPlugin.DoE</pkg> (beta version; tutorial available in Groemping 2011), which integrates 
  design of experiments functionality into the R-Commander (package "Rcmdr", Fox 2005) 
  for the benefit of those R users who cannot or do not want to do command line programming. 
  The role of package <pkg>DoE.wrapper</pkg> in that suite is to wrap 
  functionality from other packages into the input and output structure of the package suite 
  (so far for response surface designs with package <pkg>rsm</pkg> (cf. also below), 
  design of computer experiments with packages <pkg>lhs</pkg> and <pkg>DiceDesign</pkg> (cf. also below), 
  and , and D-optimal designs with package <pkg>AlgDesign</pkg> (cf. also above). 
  </li>
  
   
  <li>Package <pkg>dae</pkg> provides various utility functions around experimental design 
  and R factors, e.g. a randomization routine that can handle various nested structures 
  (according to Bailey 1981) and functions for combining several factors into one 
  or dividing one factor into several factors. 
  Furthermore, the package provides features for post-processing 
  objects returned by the <code>aov()</code> function, e.g. extraction of Yates effects 
  for 2-level experiments.</li>
  <li>Package <pkg>daewr</pkg> accompanies the book <i>Design and Analysis of Experiments with R</i> 
  by Lawson (2014) and does not only provide data sets from the book but also some standalone functionality 
  that is not available elsewhere in R, e.g. definitive screening designs.</li>
  <li>Package <pkg>OPDOE</pkg> accompanies the book <i>Optimal Experimental Design with
   R</i> by Rasch et al. (2011). It has some interesting sample size estimation functionality, 
   but is almost unusable without the book (the first edition of which I would not recommend buying).</li> 
  <li>Package <pkg>blockTools</pkg> assigns units to blocks in order to end up with homogeneous sets
  of blocks in case of too small block sizes; package <pkg>blocksdesign</pkg> permits 
  the creation of nested block structures.</li>
  <li>There are several packages for determining sample sizes in experimental contexts, 
  some of them quite general, others very specialized. All of these are mentioned here: 
  packages <pkg>powerAnalysis</pkg>, <pkg>powerbydesign</pkg> and <pkg>easypower</pkg> deal with estimating 
  the power, sample size and/or effect size for factorial experiments. Package <pkg>JMdesign</pkg> 
  deals with the power for the special situation of jointly modeling longitudinal and survival data,
  package <pkg>PwrGSD</pkg> with the power for group sequential designs, 
  package <pkg>powerGWASinteraction</pkg> with the power for interactions in genome wide association studies, 
  package <pkg>ssizeRNA</pkg> with sample size for RNA sequencing experiments, and package <pkg>ssize.fdr</pkg> 
  for sample sizes in microarray experiments (requesting a certain power while limiting the false discovery rate).</li>
</ul>

  <h3>Experimental designs for industrial experiments</h3>
  <p>Some further packages especially handle designs for industrial experiments
  that are often highly fractionated, intentionally confounded and have few extra degrees
  of freedom for error.</p> 
  
  <p>Fractional factorial 2-level designs are particularly important in industrial 
  experimentation. 
  </p>
  <ul>
  <li>Package <pkg>FrF2</pkg> (Groemping 2014) is the most comprehensive R package for 
  their creation. It generates regular Fractional Factorial 
  designs for factors with 2 levels as well as Plackett-Burman type screening designs. 
  Regular fractional factorials default to maximum resolution minimum aberration designs 
  and can be customized in various ways, supported by an 
  incorporated catalogue of designs (including the designs catalogued by Chen, Sun and Wu 1993, 
  and further larger designs catalogued in Block and Mee 2005 and Xu 2009; 
  the additional package <pkg>FrF2.catlg128</pkg> provides a very large complete catalogue 
  for resolution IV 128 run designs with up to 23 factors for special purposes). 
  Analysis-wise, <pkg>FrF2</pkg> provides simple graphical analysis tools (normal and half-normal effects plots 
  (modified from <pkg>BsMD</pkg>, cf. below), main effects 
  plots and interaction plot matrices similar to those in Minitab software, and a cube 
  plot for the combinations of three factors). It can also show the alias structure 
  for regular fractional factorials of 2-level factors, regardless whether they have been 
  created with the package or not.<br/> 
  Fractional factorial 2-level plans can also be created by other R packages, 
  namely <pkg>BHH2</pkg> and <pkg>qualityTools</pkg> (but do not use function pbDesign from
  version 1.54 of that package!), or with a little bit more complication
	by packages <pkg>conf.design</pkg>, <pkg>planor</pkg> or <pkg>AlgDesign</pkg>.
  Package <pkg>oapackage</pkg> allows to generate (not necessarily orthogonal) 
  optimal fractional factorial 2-level designs with the possibility to prioritize 
  main effects higher than interactions. Package <pkg>ALTopt</pkg> provides optimal designs 
  for accelerated life testing.</li>
  
  <li>Package <pkg>BHH2</pkg> accompanies the 2nd edition of the book by Box, Hunter and Hunter 
  and provides various of its data sets. It can generate full and fractional factorial 
  two-level-designs from a number of factors and a list of defining relations 
  (function <code>ffDesMatrix()</code>, less comfortable than package FrF2). 
  It also provides several functions for analyzing data from 2-level factorial 
  experiments: The function anovaPlot assesses effect sizes relative to residuals, and
  the function <code>lambdaPlot()</code> assesses the effect of Box-Cox transformations on
  statistical significance of effects. </li>
  <li><pkg>BsMD</pkg> provides Bayesian charts as
  proposed by Box and Meyer (1986) as well as effects plots (normal, half-normal and
  Lenth) for assessing which effects are active in a fractional factorial experiment
  with 2-level factors; package <pkg>OBsMD</pkg> provides the functionality for follow-
  up experiments for resolving ambiguities after applying the Bayesian analysis of package 
  <pkg>BsMD</pkg>. </li>
  <li>Package <pkg>pid</pkg> accompanies an online book by Dunn (2010-2016) and also makes 
  heavy use of the Box, Hunter and Hunter book; it provides various data sets, which are 
  mostly from fractional factorial 2-level designs.</li>
  </ul>
   
  <p>
  Apart from tools for planning and analysing factorial designs, R also offers support for 
  response surface optimization for quantitative factors (cf. e.g. Myers and Montgomery 1995): 
  </p>
  <ul>
  <li>Package <pkg>rsm</pkg> supports sequential 
  optimization with first order and second order response surface models (central composite 
  or Box-Behnken designs), offering 
  optimization approaches like steepest ascent and visualization of the response 
  function for linear model objects. Also, coding for response surface investigations is 
  facilitated. </li>
  <li>Package <pkg>DoE.wrapper</pkg> enhances design creation from package <pkg>rsm</pkg> 
  with the possibilities of automatically choosing the cube portion of central 
  composite designs and of augmenting 
  an existing (fractional) factorial 2-level design with a star portion. </li>
  <li>Package <pkg>OptimaRegion</pkg> provides functionality for inspecting the optimal 
  region of a response surface for quadratic polynomials and thin-plate spline models 
  and can compute a confidence interval for the distance between two optima.</li>
  <li>Package <pkg>Vdgraph</pkg> implements a variance dispersion graph (Vining 1993) for response 
  surface designs created by package <pkg>rsm</pkg>. Packages <pkg>VdgRsm</pkg> and <pkg>vdg</pkg>
  provide similar functionality with more variety.</li>
  <li>Package <pkg>qualityTools</pkg> can also create central composite designs 	 
	   and can visualize response surfaces.</li>
  <li>Package <pkg>EngrExpt</pkg> provides a collection of data sets from the book 
  <i>Introductory Statistics for Engineering Experimentation</i> by Nelson, Coffin and Copeland (2003).</li>
  </ul>
  
  <p>In some industries, mixtures of ingredients are important; these require special designs, 
  because the quantitative factors have a fixed total. 
  Mixture designs are handled by packages <pkg>AlgDesign</pkg> (function <code>gen.mixture</code>, 
  lattice designs), <pkg>qualityTools</pkg> (function <code>mixDesign</code>,
  lattice designs and simplex centroid designs), and <pkg>mixexp</pkg> (several small functions for simplex centroid, 
  simplex lattice and extreme vertices designs as well as for plotting). </p>

  <p>Occasionally, supersaturated designs can be useful. 
  The two small packages <pkg>mkssd</pkg> and <pkg>mxkssd</pkg> provide fixed level and mixed level 
  k-circulant supersaturated designs.</p>

  <h3>Experimental designs for computer experiments</h3>
  <p>Computer experiments with quantitative factors require special types of 
  experimental designs: it is often possible to include many different 
  levels of the factors, and replication will usually not be beneficial. Also, the 
  experimental region is often too large to assume that a linear or quadratic model adequately 
  represents the phenomenon under investigation. Consequently, it is desirable to fill 
  the experimental space with points as well as possible (space-filling designs) in such 
  a way that each run provides additional information even if some factors turn out to be 
  irrelevant.
  The <pkg>lhs</pkg> package provides latin hypercube designs for this purpose. 
  Furthermore, the package provides ways to analyse such computer experiments with
  emphasis on what follow-up experiments to conduct. Another package with similar orientation 
  is the <pkg>DiceDesign</pkg> package, which adds further ways to construct space-filling 
  designs and some measures to assess the quality of designs for computer experiments. The 
  package <pkg>DiceKriging</pkg> provides the kriging methodology which is often used for 
  creating meta models from computer experiments, the package <pkg>DiceEval</pkg> creates 
  and evaluates meta models (among others Kriging ones), and the package <pkg>DiceView</pkg> 
  provides facilities for viewing sections of multidimensional meta models.</p>
  
  <p>Package <pkg>MaxPro</pkg> provides maximum projection designs as introduced by 
  Joseph, Gul and Ba(2015). Package <pkg>SLHD</pkg> provides optimal sliced latin 
  hypercube designs according to Ba et al. (2015)
  Package <pkg>simrel</pkg> allows creation of designs for 
  computer experiments according to the Multi-level binary replacement (MBR) strategy 
  by Martens et al. (2010). Package <pkg>minimaxdesign</pkg> provides 
  minimax designs and minimax projection designs according to Mak and Joseph (2016).
  </p>
  
  <p>Package <pkg>tgp</pkg> is another package dedicated to planning and analysing 
  computer experiments. Here, emphasis is on Bayesian methods. 
  The package can for example be used with various kinds of (surrogate) models for 
  sequential optimization, e.g. with an expected improvement criterion for optimizing a noisy 
  blackbox target function. Packages <pkg>plgp</pkg> and <pkg>dynaTree</pkg> enhance the 
  functionality offered by <pkg>tgp</pkg> with particle learning facilities and learning for 
  dynamic regression trees.</p>
  
  <p>Package <pkg>BatchExperiments</pkg> is also designed for computer 
  experiments, in this case specifically for experiments with algorithms to be run 
  under different scenarios. The package is described in a technical report by 
  Bischl et al. (2012).</p>

  <h3>Experimental designs for clinical trials</h3>
  <p>This task view only covers specific design of experiments packages; there may be some 
  grey areas. Please, also consult the <a href="ClinicalTrials.html">ClinicalTrials</a> task view. 
  </p>
  <ul>
  <li>Package <pkg>experiment</pkg> contains tools for clinical experiments,
  e.g., a randomization tool, and it provides a few special analysis options for clinical
  trials.</li> 
  <li>Package <pkg>ThreeArmedTrials</pkg> provides design and analysis tools for three-armed 
  superiority or non-inferiority trials. Beside the standard functionality, the package 
  includes the negative Binomial response situation discussed in Muetze et al. (2016).</li>  
  <li>Package <pkg>gsDesign</pkg> implements group sequential designs, package <pkg>GroupSeq</pkg> 
  gives a GUI for probability spending in such designs,  
  package <pkg>OptGS</pkg> near-optimal balanced group sequential designs. 
  Package <pkg>gsbDesign</pkg> evaluates operating characteristics for group sequential Bayesian designs.
  Package <pkg>gset</pkg> handles group sequential equivalence testing. Package <pkg>seqDesign</pkg> 
  handles group sequential two-stage treatment efficacy trials with time-to-event endpoints.</li> 
  <li>Package <pkg>asd</pkg> implements adaptive sequential designs.</li> 
  <li>Package <pkg>OptInterim</pkg> is for two- and three-stage designs for longterm binary endpoints.</li> 
  <li>Packages <pkg>bcrm</pkg> and <pkg>crmPack</pkg> offer Bayesian CRM designs.</li>
  <li>Package <pkg>MAMS</pkg> offers designs for multi-arm multi stage studies, <pkg>BayesMAMS</pkg> provides a 
  Bayesian sample size calculations for these.</li>
  <li>Package <pkg>BOIN</pkg> provides Bayesian optimal interval designs, which are used in phase I clinical 
  trials for finding the maximum tolerated dose. </li>
  <li>The <pkg>DoseFinding</pkg> package provides functions for the design and analysis 
  of dose-finding experiments (for example pharmaceutical Phase II clinical trials); 
  it combines the facilities of the "MCPMod" package (maintenance discontinued; 
  described in Bornkamp, Pinheiro and Bretz 2009) with a special type of optimal designs for 
  dose finding situations (MED-optimal designs, or D-optimal designs, or a mixture of both; 
  cf., Dette et al. 2008).</li> 
  <li>Package <pkg>VNM</pkg> provides multi-objective optimal designs for simultaneously optimizing 
  inference about the shape of the dose-response curve, ED50 and minimum effective dose (MED) for certain classes of 
  logistic models. </li>
  <li>Package <pkg>TEQR</pkg> 
  provides toxicity equivalence range designs (Blanchard and Longmate 2010) for phase I clinical trials, 
  package <pkg>pipe.design</pkg> so-called <i>product of independent beta probabilities dose escalation</i> (PIPE) 
  designs for phase I. Package <pkg>dfpk</pkg> implements a Bayesian dose-finding design 
  using pharmacokinetics for phase I trials. Package <pkg>dfcrm</pkg> provides designs 
  for classical or TITE continual reassessment trials in phase I.</li> 
  <li>Packages <pkg>dfcomb</pkg> and <pkg>dfmta</pkg> provide 
  phase I/II adaptive dose-finding designs for 
  combination studies or single-agent molecularly targeted agent, respectively. 
  Packages <pkg>ph2bayes</pkg> and <pkg>ph2bye</pkg> are concerned with 
  Bayesian single arm phase II trials, and package <pkg>binseqtest</pkg> handles sequential single arm binary response trials. </li>
  <li>Package <pkg>sp23design</pkg>	claims to offer seamless integration of phase II to III.</li>
    </ul>
  
  <h3>Experimental designs for special purposes</h3>
  <p>Various further packages handle special situations in experimental design:</p>
  <ul>
  <li>Package <pkg>desirability</pkg> provides ways to combine several target criteria into a desirability function in order to simplify
  multi-criteria analysis; desirabilities are also offered as part of package <pkg>qualityTools</pkg>.</li> 
  <li><pkg>osDesign</pkg> designs studies nested in observational studies, <pkg>designmatch</pkg> can also be useful for this purpose.</li>
  <li><pkg>qtlDesign</pkg> is for quantitative trait locus designs,</li>
  <li><pkg>toxtestD</pkg> creates optimal designs for binary toxicity tests,</li>
  <li><pkg>hiPOD</pkg> provides optimal designs for pooled next generation sequencing experiments,</li>
  <li><pkg>designGG</pkg> creates optimal designs for genetical genomics experiments (see Li et al. 2009),</li>
  <li>Package <pkg>docopulae</pkg> implements optimal designs for copula models according to 
  Perrone and Mueller (2016),</li> 
  <li><pkg>optDesignSlopeInt</pkg> provides an optimal design for the estimation of the ratio of slope to intercept, and</li>
  <li><pkg>geospt</pkg> allows to optimize spatial networks of sampling points (see e.g. Santacruz, Rubiano and Melo 2014).</li>
  <li>Package <pkg>SensoMineR</pkg> contains special designs for
  sensometric studies, e.g., for the triangle test.</li>
  <li>Package <pkg>choiceDes</pkg> creates 
  choice designs with emphasis on discrete choice models and MaxDiff functionality; it is based on optimal designs. 
  Package <pkg>support.CEs</pkg> provides tools for creating stated choice designs 
  for market research investigations, based on orthogonal arrays. </li>
  </ul>
  
<h3>Key references for packages in this task view</h3>
  <ul>
    <li>Atkinson, A.C. and Donev, A.N. (1992). <i>Optimum Experimental Designs.</i>
    Oxford: Clarendon Press.</li>
    
    <li>Atkinson, A.C., Donev, A.N. and Tobias, R.D. (2007). <i>Optimum Experimental Designs, with SAS.</i>
    Oxford University Press, Oxford.</li>
    
     <li>Ba,S., Brenneman, W.A. and Myers, W.R. (2015). Optimal Sliced Latin Hypercube Designs. 
    <i>Technometrics</i> <b>57</b> 479-487.</li>

    <li>Bailey, R.A. (1981). A unified approach to design of experiments. 
    <i>Journal of the Royal Statistical Society, Series A</i>
    <b>144</b> 214-223.</li>

    <li>Ball, R.D. (2005). Experimental Designs for Reliable Detection of Linkage
    Disequilibrium in Unstructured Random Population Association Studies. <i>Genetics</i>
    <b>170</b> 859-873.</li>

    <li>Bischl, B., Lang, M., Mersmann, O., Rahnenfuehrer, J. and Weihs, C. (2012). 
    <a href="http://www1.beuth-hochschule.de/FB_II/reports/Report-2011-004.pdf">
    Computing on high performance clusters with R: Packages BatchJobs and 
    BatchExperiments</a>. <i>Technical Report 1/2012</i>, TU Dortmund, Germany.</li>

    <li>Blanchard, M.S. and Longmate, J.A. (2010). 
    Toxicity equivalence range design (TEQR): A practical Phase I design. 
    <i>Contemporary Clinical Trials</i> doi:10.1016/j.cct.2010.09.011.</li>

    <li>Block, R. and Mee, R. (2005). Resolution IV Designs with 128 Runs. 
    <i>Journal of Quality Technology</i> <b>37</b> 282-293. </li>
    
    <li>Bornkamp B., Pinheiro J. C., and Bretz, F. (2009). 
    <a href="http://www.jstatsoft.org/v29/i07/paper">MCPMod: An R Package for the Design and Analysis of Dose-Finding Studies</a>. 
    <i>Journal of Statistical Software</i> <b>29</b>(7) 1-23.</li>
    
    <li>Box G. E. P, Hunter, W. C. and Hunter, J. S. (2005). <i>Statistics for Experimenters</i>
    (2nd edition). New York: Wiley.</li>
    
    <li>Box, G. E. P and R. D. Meyer (1986). An Analysis for Unreplicated Fractional
    Factorials. <i>Technometrics</i> <b>28</b> 11-18.</li>

    <li>Box, G. E. P and R. D. Meyer (1993). Finding the Active Factors in Fractionated Screening
    Experiments. <i>Journal of Quality Technology</i> <b>25</b> 94-105.</li>

    <li>Chasalow, S., Brand, R. (1995). Generation of Simplex Lattice Points. 
    <i>Journal of the Royal Statistical Society, Series C </i> <b>44</b> 534-545.</li> 

    <li>Chen, J., Sun, D.X. and Wu, C.F.J. (1993). A catalogue of 2-level and 3-level orthogonal arrays. 
    <i>International Statistical Review</i> <b>61</b> 131-145.</li>

    <li>Consonni, G. and Deldossi, L. (2015), "Objective Bayesian model discrimination in follow-up 
    experimental designs" DOI 10.1007/s11749-015-0461-3. TEST.</li>
    
    <li>Collings, B. J. (1989). Quick Confounding. <i>Technometrics</i> <b>31</b> 107-110.</li>

    <li>Cornell, J. (2002). <i>Experiments with Mixtures</i>. Third Edition. Wiley.</li> 
    
    <li>Daniel, C. (1959). Use of Half Normal Plots in Interpreting Two Level Experiments.
    <i>Technometrics</i> <b>1</b> 311-340.</li>
    
    <li>Derringer, G. and Suich, R. (1980). Simultaneous Optimization of Several Response Variables.
    <i>Journal of Quality Technology</i> <b>12</b> 214-219.</li>

    <li>Dette, H., Bretz, F., Pepelyshev, A. and Pinheiro, J. C. (2008). 
    Optimal Designs for Dose Finding Studies. <i>Journal of the American Statisical Association</i> <b>103</b> 
    1225-1237.</li>

    <li>Dette, H., Melas, V.B. and Shpilev, P. (2013). Robust T-optimal discriminating designs.  
    <i>The Annals of Statistics</i> <b>41</b> 1693-1715.</li>
    
    <li>Dette H., Melas V.B. and Guchenko R. (2014). Bayesian T-optimal discriminating designs. 
    <a href="http://arxiv.org/abs/1412.2548">ArXiv link</a>. </li>
    
    <li>Dunn, K. (2010-2016). <i>Process Improvement Using Data</i>. <a href="http://learnche.org/pid">Online book.</a></li>
    
    <li>Federov, V.V. (1972). <i>Theory of Optimal Experiments.</i> Academic Press, New York.</li>
    
    <li>Fox, J. (2005). <a href="http://www.jstatsoft.org/v14/i09/paper">The R Commander: 
    A Basic-Statistics Graphical User Interface to R</a>. 
    <i>Journal of Statistical Software</i> <b>14</b>(9) 1-42.</li>
    
    <li>Gramacy, R.B. (2007). <a href="http://www.jstatsoft.org/v19/i09/paper">tgp: An R Package 
    for Bayesian Nonstationary, Semiparametric Nonlinear Regression and Design by Treed Gaussian Process Models</a>.
    <i>Journal of Statistical Software</i> <b>19</b>(9) 1-46.</li>

    <li>Groemping, U. (2009). <a href="http://www.R-project.org/conferences/useR-2009/slides/Gromping.pdf">
    Design of Experiments in R</a>. Presentation at UseR! 2009 in Rennes, France.</li>

    <li>Groemping, U. (2011). <a href="http://www1.beuth-hochschule.de/FB_II/reports/Report-2011-004.pdf">
    Tutorial for designing experiments using the R package RcmdrPlugin.DoE</a>. 
    <a href="http://www1.beuth-hochschule.de/FB_II/reports/welcome.htm">Reports in Mathematics, Physics and Chemistry</a>, 
    Department II, Beuth University of Applied Sciences Berlin.</li>

    <li>Groemping, U. (2014). R Package FrF2 for Creating and Analysing Fractional Factorial 2-Level Designs. 
    <i>Journal of Statistical Software</i> <b>56</b>(1) 1-56.</li> 
    
    <li>Groemping, U. and Xu, H. (2014). Generalized resolution for orthogonal arrays. 
    <i>The Annals of Statistics</i> <b>42</b> 918-939.</li>

    <li>Groemping, U. (2015). Augmented Half Normal Effects Plots in the Presence 
    of a Few Error Degrees of Freedom. <i>Quality and Reliability Engineering International</i> 
    <b>online early</b> DOI: 10.1002/qre.1842.</li> 
    
    <li>Harman R., Filova L. (2014). Computing efficient exact designs of experiments using integer 
    quadratic programming, <i>Computational Statistics and Data Analysis</i>, <b>71</b>, 1159-1167</li>

    <li>Hoaglin D., Mosteller F. and Tukey J. (eds., 1991). <i>Fundamentals of Exploratory Analysis of Variance</i>. 
    Wiley, New York.</li>      

    <li>Jones, B. and Kenward, M.G. (1989). <i>Design and Analysis of Cross-Over Trials</i>. Chapman and
    Hall, London.</li>

    <li>Johnson, M.E., Moore L.M. and Ylvisaker D. (1990). Minimax and maximin distance designs. 
    <i>Journal of Statistical Planning and Inference</i> <b>26</b> 131-148.</li>

    <li>Joseph, V. R., Gul, E., and Ba, S. (2015). Maximum Projection Designs for 
    Computer Experiments. <i>Biometrika</i> <b>102</b> 371-380.</li>
    
    <li>Kuhfeld, W. (2009). Orthogonal arrays. Website courtesy of SAS Institute Inc., accessed August 4th 2010. 
    URL <a href="http://support.sas.com/techsup/technote/ts723.html">http://support.sas.com/techsup/technote/ts723.html</a>.</li>

    <li>Kuhnert, P. and Venables, B. (2005) <i>An Introduction to R: Software for Statistical
    Modelling &amp; Computing</i>. URL <a href="../../doc/contrib/Kuhnert+Venables-R_Course_Notes.zip">http://CRAN.R-project.org/doc/contrib/Kuhnert+Venables-R_Course_Notes.zip</a>.
    (PDF document (about 360 pages) of lecture notes in combination with the data sets and R scripts)</li>
    
    <li>Kunert, J. (1998). Sensory Experiments as Crossover Studies. <i>Food Quality and Preference</i>
    <b>9</b> 243-253.</li>

    <li>Lalanne, C. (2012). R Companion to Montgomerys Design and Analysis of Experiments.
    Manuscript, downloadable at URL 
    <a href="http://www.aliquote.org/articles/tech/dae/dae.pdf">http://www.aliquote.org/articles/tech/dae/dae.pdf</a>.
    (The file accompanies the book by Montgomery 2005 (cf. below).)</li>
    
    <li>Lawson, J. (2014). <i>Design and Analysis of Experiments with R.</i> Chapman and Hall/CRC, Boca Raton.</li>
    
    <li>Lenth, R.V. (1989). Quick and Easy Analysis of Unreplicated Factorials.
    <i>Technometrics</i> <b>31</b> 469-473.</li>

    <li>Lenth, R.V. (2009). <a href="http://www.jstatsoft.org/v32/i07/paper">Response-Surface Methods in R, Using rsm</a>.
    <i>Journal of Statistical Software</i> <b>32</b>(7) 1-17.</li>
    
    <li>Y. Li, M. Swertz, G. Vera, J. Fu, R. Breitling, and R.C. Jansen. <pkg>designGG</pkg>: 
    An R-package and Web tool for the optimal design of genetical genomics experiments. 
    <i>BMC Bioinformatics</i> <b>10</b>:188</li>

    <li>Mak, S., and Joseph, V.R. (2016). Minimax designs using clustering. <i>Journal of Computational and
    Graphical Statistics</i>. In revision.</li>
    
    <li>Martens, H., Mage, I., Tondel, K., Isaeva, J., Hoy, M. and Saebo, S. (2010). 
    Multi-level binary replacement (MBR) design for computer experiments in high-dimensional 
    nonlinear systems, <i>J. Chemom.</i> <b>24</b> 748-756.</li>
    
    <li>Masoudi, E., Holling, H. and Wong, W.-K. (2016). Application of imperialist competitive algorithm 
    to find minimax and standardized maximin optimal designs. <i>Computational Statistics and Data Analysis</i>, 
    in press. DOI: 10.1016/j.csda.2016.06.014</li>

    <li>Mee, R. (2009). <i>A Comprehensive Guide to Factorial Two-Level Experimentation.</i> 
    Springer, New York.</li>

    <li>Montgomery, D. C. (2005, 6th ed.). <i>Design and Analysis of Experiments.</i> Wiley, New York.</li>
    
    <li>Muetze,T., Munk, A. and Friede, T. (2016). Design and analysis of three-arm trials with negative
    binomially distributed endpoints. <i>Statistics in Medicine</i> <b>35</b>(4) 505-521.</li>

    <li>Myers, R. H. and Montgomery, D. C. (1995). <i>Response Surface Methodology: Process and Product
    Optimization Using Designed Experiments.</i> Wiley, New York.</li>
    
    <li>Nelson, P.R., Coffin, M. and Copeland, K.A.F. (2003). <i>Introductory Statistics for Engineering 
    Experimentation.</i>Academic Press, San Diego.</li>
    
    <li>Perrone, E. and Mueller, W.G. (2016) Optimal designs for copula models, <i>Statistics</i> 
    <b>50</b>(4), 917-929. DOI: 10.1080/02331888.2015.1111892</li>

    <li>Plackett, R.L. and Burman, J.P. (1946). The design of optimum multifactorial experiments. <i>Biometrika</i> <b>33</b> 305-325.</li>

    <li>Rasch, D., Pilz, J., Verdooren, L.R. and Gebhardt, A. (2011). <i>Optimal Experimental 
    Design with R.</i> Chapman and Hall/CRC. (caution, does not live up to its title!)</li>

    <li>Rosenbaum, P. (1989). Exploratory Plots for Paired Data. <i>The American Statistician</i> <b>43</b> 108-109.</li> 

    <li>Sacks, J., Welch, W.J., Mitchell, T.J. and Wynn, H.P. (1989). Design and analysis of computer experiments.
    <i>Statistical Science</i> <b>4</b> 409-435.</li>

    <li>Santacruz, A., Rubiano, Y., Melo, C., 2014. Evolutionary optimization of spatial sampling networks 
    designed for the monitoring of soil carbon. In: Hartemink, A., McSweeney, K. (Eds.). <i>Soil Carbon.</i> 
    Series: Progress in Soil Science. (pp. 77-84). Springer, New York. </li>

    <li>Santner T.J., Williams B.J. and Notz W.I. (2003). <i>The Design and Analysis of Computer Experiments.</i>
    Springer, New York.</li>

    <li>Sen S, Satagopan JM and Churchill GA (2005). Quantitative Trait Locus Study Design from an Information
    Perspective. <i>Genetics</i> <b>170</b> 447-464.</li>
    
    <li>Stein, M. (1987). Large Sample Properties of Simulations Using Latin Hypercube Sampling.
    <i>Technometrics</i> <b>29</b> 143-151.</li>
    
    <li>Stocki, R. (2005). A Method to Improve Design Reliability Using Optimal Latin Hypercube Sampling.
    <i>Computer Assisted Mechanics and Engineering Sciences</i> <b>12</b> 87-105.</li>

    <li>Underwood, A.J. (1997). <i>Experiments in Ecology: Their Logical Design and Interpretation Using Analysis of Variance.</i>
    Cambridge University Press, Cambridge.</li>

    <li>Vikneswaran (2005). <i>An R companion to "Experimental Design".</i> 
    URL <a href="../../doc/contrib/Vikneswaran-ED_companion.pdf">http://CRAN.R-project.org/doc/contrib/Vikneswaran-ED_companion.pdf</a>.
    (The file accompanies the book "Experimental Design with Applications in Management, Engineering
    and the Sciences" by Berger and Maurer, 2002.)</li>

    <li>Vining, G. (1993). A Computer Program for Generating Variance Dispersion Graphs. 
    <i>Journal of Quality Technology</i> <b>25</b> 45-58. Corrigendum in the same volume, pp. 333-335.</li>

    <li>Xu, H. (2009). Algorithmic Construction of Efficient Fractional Factorial Designs With Large Run Sizes. 
    <i>Technometrics</i> <b>51</b> 262-277. </li>    

  </ul>
  
</info>

<packagelist>
  <pkg>acebayes</pkg>
  <pkg priority="core">agricolae</pkg>
  <pkg>agridat</pkg>
  <pkg priority="core">AlgDesign</pkg>
  <pkg>ALTopt</pkg>
  <pkg>asd</pkg>
  <pkg>BatchExperiments</pkg>
  <pkg>BayesMAMS</pkg>
  <pkg>bcrm</pkg>
  <pkg>BHH2</pkg>
  <pkg>binseqtest</pkg>
  <pkg>blockTools</pkg>
  <pkg>blocksdesign</pkg>
  <pkg>BOIN</pkg>
  <pkg>BsMD</pkg>
  <pkg>choiceDes</pkg>
  <pkg>CombinS</pkg>
  <pkg priority="core">conf.design</pkg>
  <pkg>crmPack</pkg>
  <pkg priority="core">crossdes</pkg>
  <pkg>Crossover</pkg>
  <pkg>dae</pkg>
  <pkg>daewr</pkg>
  <pkg>designGG</pkg>
  <pkg>designGLMM</pkg>
  <pkg>designmatch</pkg>
  <pkg>desirability</pkg>
  <pkg>desplot</pkg>
  <pkg>dfcomb</pkg>
  <pkg>dfcrm</pkg>
  <pkg>dfmta</pkg>
  <pkg>dfpk</pkg>
  <pkg>DiceDesign</pkg>
  <pkg>DiceEval</pkg>
  <pkg>DiceKriging</pkg>
  <pkg>DiceView</pkg>
  <pkg>docopulae</pkg>
  <pkg priority="core">DoE.base</pkg>
  <pkg priority="core">DoE.wrapper</pkg>
  <pkg>DoseFinding</pkg>
  <pkg>dynaTree</pkg>
  <pkg>easypower</pkg>
  <pkg>EngrExpt</pkg>
  <pkg>experiment</pkg>
  <pkg>ez</pkg>
  <pkg priority="core">FrF2</pkg>
  <pkg>FrF2.catlg128</pkg>
  <pkg>GAD</pkg>
  <pkg>geospt</pkg>
  <pkg>granova</pkg>
  <pkg>GroupSeq</pkg>
  <pkg>gsDesign</pkg>
  <pkg>gsbDesign</pkg>
  <pkg>gset</pkg>
  <pkg>hiPOD</pkg>
  <pkg>ibd</pkg>
  <pkg>ICAOD</pkg>
  <pkg>JMdesign</pkg>
  <pkg>lhs</pkg>
  <pkg>LDOD</pkg>
  <pkg>MAMS</pkg>
  <pkg>MaxPro</pkg>
  <pkg>minimaxdesign</pkg>
  <pkg>mixexp</pkg>
  <pkg>mkssd</pkg>
  <pkg>mxkssd</pkg>
  <pkg>oapackage</pkg>
  <pkg>OBsMD</pkg>
  <pkg>OPDOE</pkg>
  <pkg>optDesignSlopeInt</pkg>
  <pkg>OptGS</pkg>
  <pkg>OptimalDesign</pkg>
  <pkg>OptimaRegion</pkg>
  <pkg>OptInterim</pkg>
  <pkg>osDesign</pkg>
  <pkg>PGM2</pkg>
  <pkg>ph2bayes</pkg>
  <pkg>ph2bye</pkg>
  <pkg>pid</pkg>
  <pkg>pipe.design</pkg>
  <pkg priority="core">planor</pkg>
  <pkg>plgp</pkg>
  <pkg>PopED</pkg>
  <pkg>powerAnalysis</pkg>
  <pkg>powerbydesign</pkg>
  <pkg>powerGWASinteraction</pkg>
  <pkg>PwrGSD</pkg>
  <pkg>qtlDesign</pkg>
  <pkg>qualityTools</pkg>
  <pkg>RcmdrPlugin.DoE</pkg>
  <pkg>RPPairwiseDesign</pkg>
  <pkg>rodd</pkg>
  <pkg priority="core">rsm</pkg>
  <pkg>SensoMineR</pkg>
  <pkg>seqDesign</pkg>
  <pkg>simrel</pkg>
  <pkg>SLHD</pkg>
  <pkg>sp23design</pkg>
  <pkg>ssize.fdr</pkg>
  <pkg>ssizeRNA</pkg>
  <pkg>support.CEs</pkg>
  <pkg>TEQR</pkg>
  <pkg>tgp</pkg>
  <pkg>ThreeArmedTrials</pkg>
  <pkg>toxtestD</pkg>
  <pkg>vdg</pkg>
  <pkg>Vdgraph</pkg>
  <pkg>VdgRsm</pkg>
  <pkg>VNM</pkg>
</packagelist>

<links>
  <a href="http://learnche.org/pid">Dunn, K. (2010-2016). Process Improvement Using Data.</a>
  <a href="../../doc/contrib/Kuhnert+Venables-R_Course_Notes.zip"> Kuhnert, P. and Venables, B. (2005) <i>An Introduction to R: Software for Statistical Modelling &amp; Computing</i>. (~4MB)</a>
  <a href="../../doc/contrib/Vikneswaran-ED_companion.pdf"> Vikneswaran (2005). An R companion to "Experimental Design".</a>
</links>

</CRANTaskView>
