%\VignetteIndexEntry{Pairwise Multiple Comparison of Mean Rank Sums}
%\VignetteDepends{PMCMR}
%\VignetteKeywords{Nemenyi test}
%\VignettePackage{PMCMR}

\documentclass[a4paper]{scrartcl}
\usepackage{Sweave}
\usepackage[utf8]{inputenc}
\usepackage{natbib}
\usepackage{url}

%\SweaveOpts{concordance=TRUE}

\title{The Pairwise Multiple Comparison of Mean Ranks Package (PMCMR)}
\author{Thorsten Pohlert}
\date{latest revision: 2016-01-06}
\begin{document}
\bibliographystyle{jss}

\maketitle
\begin{small}
{\copyright} Thorsten Pohlert. This work is licensed under a Creative Commons License (CC BY-ND 4.0). See \url{http://creativecommons.org/licenses/by-nd/4.0/} for details. Please cite this package as: \\

T. Pohlert (2014). \textit{The Pairwise Multiple Comparison of Mean Ranks Package (PMCMR).} R package. \url{http://CRAN.R-project.org/package=PMCMR}. \\

See also \texttt{citation("PMCMR")}.
\end{small}

\tableofcontents

\section{Introduction}
The Kruskal and Wallis one-way analysis of variance by ranks 
or van der Waerden's normal score test can be employed, 
if the data do not meet the assumptions 
for one-way ANOVA. Provided that significant differences 
were detected by the omnibus test, one may be interested 
in applying post-hoc tests for pairwise multiple comparisons 
(such as Nemenyi's test, Dunn's test, Conover's test,
van der Waerden's test). Similarly, one-way ANOVA with repeated 
measures that is also referred to as ANOVA with unreplicated 
block design can also be conducted via the Friedman-Test 
or the Quade-test. The consequent post-hoc pairwise 
multiple comparison tests according to Nemenyi, Conover and Quade
are also provided in this package. Finally Durbin's test for 
a two-way balanced incomplete block design (BIBD) is also given
in this package.

\section{Comparison of multiple independent samples (One-factorial design)}
\subsection{Kruskal and Wallis test}

The linear model of a one-way layout can be written as:
\begin{equation}
 y_{i} = \mu + \alpha_{i} + \epsilon_{i} ,
\end{equation}

with $y$ the response vector, $\mu$ the global mean of the data, $\alpha_i$ the difference to the mean of the $i$-th group and $\epsilon$ the residual error. The non-parametric alternative is the Kruskal and Wallis test. It tests the null hypothesis, that each of the $k$ samples belong to the same population ($H_0: \bar{R}_{i.} = (n + 1 ) / 2$). First, the response vector $y$ is transformed into ranks with increasing order. In the presence of sequences with equal values (i.e. ties), mean ranks are designated to the corresponding realizations. Then, the test statistic can be calculated according to Eq. \ref{eq:kruskal}:

\begin{equation}\label{eq:kruskal}
\hat{H} = \left[ \frac{12}{n\left(n + 1 \right)}\right]  \left[\sum_{i=1}^k \frac{R_i^2}{n_i} \right] - 3 \left(n + 1\right)
\end{equation}

with $n = \sum_i^k n_i$ the total sample size, $n_i$ the number of data of the $i$-th group and $R_i^2$ the squared rank sum of the $i$-th group. In the presence of many ties, the test statistics $\hat{H}$ can be corrected using Eqs. \ref{eq:tie} and \ref{eq:hcor}

\begin{equation}\label{eq:tie}
C = 1 - \frac{\sum_{i=1}^{i=r} \left(t_i^3 - t_i\right)}{n^3 - n},
\end{equation}

with $t_i$ the number of ties of the $i$-th group of ties.

\begin{equation}\label{eq:hcor}
\hat{H}^{*} = \hat{H} / C
\end{equation}

The Kruskal and Wallis test can be employed as a global test.  As the test statistic $\bar{H}$ is approximately $\chi^2$-distributed, the null hypothesis is withdrawn, if $\hat{H} > \chi^2_{k-1;\alpha}$. It should be noted, that the tie correction has only a small impact on the calculated statistic and its consequent estimation of levels of significance.

\subsection{Kruskal-Wallis -- post-hoc tests after Nemenyi}

Provided, that the globally conducted Kruskal-Wallis test indicates significance (i.e. $H_0$ is rejected, and $H_A:$ 'at least on of the $k$ samples does not belong to the same population' is accepted), one may be interested in identifying which group or groups are significantly different. The number of pairwise contrasts or subsequent tests that need to be conducted is $m = k \left( k - 1 \right) / 2$ to detect the differences between each group. Nemenyi proposed a test that originally based on rank sums and the application of the \emph{family-wise error} method to control Type I error inflation, if multiple comparisons are done. The Tukey and Kramer approach uses mean rank sums and can be employed for equally as well as unequally sized samples without ties \citep[p. 397]{Sachs_1997}. The null hypothesis $H_0: \bar{R}_i = \bar{R}_j$ is rejected, if a critical absolute difference of mean rank sums is exceeded.

\begin{equation}\label{eq:tk}
\left| \bar{R}_i - \bar{R}_j \right| > \frac{q_{\infty;k;\alpha}}{\sqrt{2}}   \sqrt{ \left[\frac{n \left( n + 1 \right)}{12} \right] 
  \left[\frac{1}{n_i} + \frac{1}{n_j}\right]},
\end{equation}

where $q_{\infty;k;\alpha}$ denotes the upper quantile of the studentized range distribution. % Although these quantiles can not be computed analytically, as $df = \infty$, a good approximation is to set $df$ very large: such as $q_{1000000;k;\alpha} \sim q_{\infty;k;\alpha}$. 
This inequality (\ref{eq:tk}) leads to the same critical differences of rank sums ($\left| R_i - R_j \right|$) when multiplied with $n$ for $\alpha = [0.1, 0.5, 0.01]$, as reported in the tables of \citet[pp. 29--31]{Wilcoxon_Wilcox_1964}. In the presence of ties the approach presented by \citet[p. 395]{Sachs_1997} can be employed (\ref{eq:chisq}), provided that $(n_i, n_j, \ldots, n_k \geq 6$) and $k \geq 4$.:

\begin{equation}\label{eq:chisq}
\left| \bar{R}_i - \bar{R}_j \right| > \sqrt{\frac{1}{C} ~ \chi^{2}_{k-1;\alpha} \left[\frac{n \left( n + 1 \right)}{12} \right] 
  \left[\frac{1}{n_i} + \frac{1}{n_j}\right]},
\end{equation}

where $C$ is given by Eq. \ref{eq:tie}. The function \texttt{posthoc.kruskal.nemenyi.test} does not evaluate the critical differences as given by Eqs. \ref{eq:tk} and \ref{eq:chisq}, but calculates the corresponding level of significance for the estimated statistics $q$ and $\chi^{2}$, respectively.

In the special case, that several treatments shall only be tested against one control experiment, the number of tests reduces to $m = k - 1$. This case is given in section \ref{control}.

\subsection{Examples using  \texttt{posthoc.kruskal.nemenyi.test}}

The function \texttt{kruskal.test} is provided with the library \texttt{stats} \citep{RCoreTeam}. The data-set \texttt{InsectSprays} was derived from a one factorial experimental design and can be used for demonstration purposes. Prior to the test, a visualization of the data (Fig \ref{Fig:boxplot}) might be helpful:

\begin{figure}[tbh]
%  \begin{center}
<<fig=TRUE ,echo=FALSE >>=
library("graphics")
boxplot(count ~ spray, data=InsectSprays)
@
\caption{Boxplot of the \texttt{InsectSprays} data set.}\label{Fig:boxplot}
%\end{center}
\end{figure}

Based on a visual inspection, one can assume that the insecticides \emph{A, B, F} differ from \emph{C, D, E}. The global test can be conducted in this way:

<<>>=
kruskal.test(count ~ spray, data=InsectSprays)
@

As the Kruskal-Wallis Test statistics is highly significant ($\chi^{2}(5) = 54.69, p < 0.01$), the null hypothesis is rejected. Thus, it is meaningful to apply post-hoc tests with the function \texttt{posthoc.kruskal.nemenyi.test}.

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.kruskal.nemenyi.test(x=count, g=spray, dist="Tukey")
@

The test returns the lower triangle of the matrix that contains the p-values of the pairwise comparisons. Thus $|\bar{R}_A - \bar{R}_B|: n.s.$, but $|\bar{R}_A - \bar{R}_C|: p < 0.01$. Since PMCMR-1.1 there is a formula method included. Thus the test can also be conducted in the following way:
<<>>=
posthoc.kruskal.nemenyi.test(count ~ spray, data=InsectSprays, dist="Tukey")
@

As there are ties present in the data, one may also conduct the Chi-square approach:

<<>>=
(out <- posthoc.kruskal.nemenyi.test(x=count, g=spray, dist="Chisquare"))
@

which leads to different levels of significance, but to the same test decision.  The arguments of the returned object of class \texttt{PMCMR} can be further explored. The statistics, in this case the $\chi^{2}$ estimations, can be taken in this way:

<<>>=
print(out$statistic)
@ 

%The test results can be aligned into a summary table as it is common in scientific articles. However, there is not yet a function included in the package \texttt{PMCMR}. Therefore, Table \ref{tab:summary} was manually created.

%\begin{table}[bh]
%\caption{Mean rank sums of insect counts ($\bar{R}_i$) after the application of insecticides (Group). Different letters indicate significant differences ($p < 0.05$) according to the Tukey-Kramer-Nemenyi post-hoc test. The global test according to Kruskal and Wallis indicated significance ($\chi^{2}(5) = 54.69, p < 0.01$). }\label{tab:summary}
%\begin{tabular}{ccl}
%\hline
%Group & $\bar{R}_i$ &  \\
%\hline
%C & 11.46 & a \\ 
%E & 19.33 & a \\
%D & 25.58 & a \\
%A & 52.17 & b \\
%B & 54.83 & b \\
%F & 55.62 & b \\
%\hline
%\end{tabular}
%\end{table}

\subsection{Kruskal-Wallis -- post-hoc test after Dunn}
\label{Dunn}
\citet{Dunn_1964} has proposed a test for multiple comparisons of rank sums based on the $z$-statistics of the standard normal distribution. The null hypothesis ($H0$), the probability of observing a randomly selected value from the first group that is larger than a randomly selected value from the second group equals one half, is rejected, if a critical absolute difference of mean rank sums is exceeded:

\begin{equation}\label{eq:dunn}
\left| \bar{R}_i - \bar{R}_j \right| > z_{1-\alpha/2*} ~ {\sqrt{ \left[\frac{n \left( n + 1 \right)}{12} - B \right] 
  \left[\frac{1}{n_i} + \frac{1}{n_j}\right]}},
\end{equation}

with $z_{1-\alpha/2*}$ the value of the standard normal distribution for a given adjusted $\alpha/2*$ level depending on the number of tests conducted and $B$ the correction term for ties, which was taken from \citet{Glantz_2012} and is given by Eq. \ref{eq:B}:

\begin{equation}\label{eq:B}
  B = \frac{\sum_{i=1}^{i=r} \left(t_i^3 - t_i\right)}{12 \left(n - 1 \right)}
\end{equation}

The function \texttt{posthoc.kruskal.dunn.test} does not evaluate the critical differences as given by Eqs. \ref{eq:dunn}, but calculates the corresponding level of significance for the estimated statistics $z$, as adjusted by any method implemented in \texttt{p.adjust} to account for Type I error inflation. It should be noted that \citet{Dunn_1964} originally used a Bonferroni adjustment of $p$-values. For this specific case, the test is sometimes referred as to the Bonferroni-Dunn test.

\subsection{Example using  \texttt{posthoc.kruskal.dunn.test}}

We can go back to the example with InsectSprays.

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.kruskal.dunn.test(x=count, g=spray, p.adjust.method="none")
@

The test returns the lower triangle of the matrix that contains the $p$-values of the pairwise comparisons. As \texttt{p.adjust.method="none"}, the $p$-values are not adjusted. Hence, there is a Type I error inflation that leads to a false positive discovery rate. This can be solved by applying e.g. a Bonferroni-type adjustment of $p$-values.

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.kruskal.dunn.test(x=count, g=spray, p.adjust.method="bonferroni")
@



\subsection{Kruskal-Wallis -- post-hoc test after Conover}
\citet{Conover_Iman_1979} proposed a test that aims at having a higher test power than the tests given by inequalities \ref{eq:tk} and \ref{eq:chisq}:

\begin{equation}\label{eq:conover}
\left| \bar{R}_i - \bar{R}_j \right| > t_{1-\alpha/2; n-k}   \sqrt{s^2 \left[\frac{n-1-\hat{H}^{*}}{n-k}  \right] 
  \left[\frac{1}{n_i} + \frac{1}{n_j}\right]},
\end{equation}

with $\hat{H}^{*}$ the tie corrected Kruskal-Wallis statistic according to Eq. \ref{eq:hcor} and $t_{1-\alpha/2; n-k}$ the $t$-value of the student-t-distribution. The variance $s^2$ is given in case of ties by:

\begin{equation}\label{eq:conover.s2}
  s^2 = \frac{1}{n-1} \left[\sum R_i^2 - n \frac{\left(n + 1 \right)^2 }{4}\right]
\end{equation}

The variance $s^2$ simplifies to $n \left(n+1\right)/12$, if there are no ties present. Although \citet{Conover_Iman_1979} did not propose an adjustment of $p$-values, the function \texttt{posthoc.kruskal.conover.test} has implemented methods for $p$-adjustment from the function \texttt{p.adjust}.

\subsection{Example using  \texttt{posthoc.kruskal.conover.test}}

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.kruskal.conover.test(x=count, g=spray, p.adjust.method="none")
@

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.kruskal.conover.test(x=count, g=spray, p.adjust.method="bonferroni")
@


\subsection{Dunn's multiple comparison test with one control}
\label{control}
Dunn's test (see section \ref{Dunn}), can also be applied for multiple comparisons with one control \citep{Siegel_Castellan_1988}: 

\begin{equation}\label{eq:dunnc}
\left| \bar{R}_0 - \bar{R}_j \right| > z_{1-\alpha/2*} ~ {\sqrt{ \left[\frac{n \left( n + 1 \right)}{12} - B \right] 
  \left[\frac{1}{n_0} + \frac{1}{n_j}\right]}},
\end{equation}

where $\bar{R}_0$ denotes the mean rank sum of the control experiment. In this case the number of tests is reduced to $m = k - 1$, which changes the $p$-adjustment according to Bonferroni (or others). The function \texttt{dunn.test.control} employs this test, but \textbf{the user need to be sure that the control is given as the first level in the group vector}.

\subsection{Example using \texttt{dunn.test.control}}
We can use the \texttt{PlantGrowth} dataset, that comprises data with dry matter weight of yields with one control experiment (i.e. no treatment) and to different treatments. In this case we are only interested, whether the treatments differ significantly from the control experiment.

<<>>=
require(stats) 
data(PlantGrowth)
attach(PlantGrowth)
kruskal.test(weight, group)
dunn.test.control(x=weight,g=group, p.adjust="bonferroni")
@ 

According to the global Kruskal-Wallis test, there are significant differences between the groups, $\chi^2(2) = 7.99, p < 0.05$. However, the Dunn-test with Bonferroni adjustment of $p$-values shows, that there are no significant effects.

If one may cross-check the findings with ANOVA and multiple comparison with one control using the LSD-test, he/she can do the following:
<<>>=
summary.lm(aov(weight ~ group))
@ 

The last line provides the statistics for the global test, i.e. there is a significant treatment effect according to one-way ANOVA, $F(2,27) = 4.85, p < 0.05, \eta^2 = 0.264$. The row that starts with \texttt{Intercept} gives the group mean of the control, its standard error, the t-value for testing $H0: \mu = 0$ and the corresponding level of significance. The following lines provide the difference between the averages of the treatment groups with the control, where $H0: \mu_0 - \mu_j = 0$. Thus the \texttt{trt1} does not differ significantly from the \texttt{ctr}, $t = -1.331, p = 0.194$. There is a significant difference between \texttt{trt2} and \texttt{ctr} as indicated by $t = 1.772, p < 0.1$.% Consequently, the test decision of this example is the same, as if \texttt{dunn.test.control} is used with Bonferroni adjustment of $p$-values.  

\subsection{van der Waerden test}
The van der Waerden test can be used as an alternative to the Kruskal-Wallis test, if the data to not meet the requirements for ANOVA \citep{Conover_Iman_1979}. Let the Kruskal-Wallis ranked data denote $R_{i,j}$, then the normal scores $A_{i,j}$ are derived from the standard normal distribution according to Eq. \ref{eq:ztrans}.

\begin{equation}\label{eq:ztrans}
A_{i,j} = \phi^{-1} \left(\frac{R_{i,j}}{n+1}\right) 
\end{equation}

Let the sum of the $i$-th score denote $A_j$. The variance $S^2$ is calculated as given in Eq. \ref{eq:vWdvar}.

\begin{equation}\label{eq:vWdvar}
S^2 = \frac{1}{n-1} \sum A_{i,j}^2 
\end{equation}

Finally the test statistic is given by Eq. \ref{eq:vWdstat}.

\begin{equation}\label{eq:vWdstat}
  T = \frac{1}{S^2} \sum_{j=1}^k \frac{A_j^2}{n_j}
\end{equation}

The test statistic $T$ is approximately $\chi^2$-distributed and tested for significance on a level of $1 - \alpha$ with $df= k-1$.

\subsection{Example using \texttt{vanWaerden.test}}

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
vanWaerden.test(x=count, g=spray)
@


\subsection{post-hoc test after van der Waerden for multiple pairwise comparisons}
Provided that the global test according to van der Waerden indicates significance, multiple comparisons can be done according to the inequality \ref{eq:multvWd}.

\begin{equation}\label{eq:multvWd}
\| \frac{A_i}{n_i} - \frac{A_j}{n_j}  \| >  t_{1-\alpha/2*; n-k} \sqrt{S^2 ~~ \frac{n-1-T}{n-k} \left(\frac{1}{n_i} + \frac{1}{n_j} \right) }
  \end{equation}

The test given in \citet{Conover_Iman_1979} does not adjust $p$-values. However, the function has included the methods for $p$-value adjustment as given by \texttt{p.adjust}. 
\subsection{Example using \texttt{posthoc.vanWaerden.test}}

<<>>=
require(PMCMR)
data(InsectSprays)
attach(InsectSprays)
posthoc.vanWaerden.test(x=count, g=spray, p.adjust.method="none")
@

\section{Test of $k$ independent samples against an ordered alternative}
\subsection{Jonckheere-Terpstrata test}

For testing $k$ independent samples against an ordered alternative (e.g. $k$ groups with increasing doses of treatment), the test according to \citet{Jonckheere_1954} can be employed. Both, the null and alternative hypothesis can be formulated as population medians ($\theta_i$) for $k$ populations ($k > 2$). Thus the null hypothesis, $H0 : \theta_1 = \theta_2 = \ldots = \theta_k$ is tested e.g. in the one-sided case against the alternative, $HA: \theta_1 \leq \theta_2 \leq \ldots \leq \theta_k$, with at least one strict inequality.

The statistic $J$ is calculated according to Eq. \ref{eq:jonck}.

\begin{equation}\label{eq:jonck}
  J = \sum_{i=1}^{k-1} \sum_{j = i + 1}^k U_{ij}
\end{equation}

where $U_{ij}$ is defined as:

\begin{equation}
  U_{ij} = \sum_{s=1}^{n_i} \sum_{t=1}^{n_j} \psi \left(X_{jt} - X_{is} \right)
\end{equation}

with

\begin{equation}
  \psi \left( u \right) = \left\{ \begin{array}{c c c}
         1 & \forall & u > 0 \\
         1/2 & \forall & u = 0 \\
         0 & \forall & u < 0
    \end{array}\right.
\end{equation}

The mean $\mu_J$ is given by

\begin{equation}
  \mu_J = \frac{N^2 - \sum_{i=1}^k n_i^2}{4}
\end{equation}

and the variance $\sigma_J$ is defined as

\begin{equation}
  \sigma_J = \sqrt{\frac{N^2 \left(2 N + 3 \right) - 
    \sum_{i=1}^k n_i^2 \left( 2 n_i + 3 \right)}{72}}.
\end{equation}

The $\hat{z}$-value of the standard normal distribution is calculated as:

\begin{equation}
  \hat{z} = \frac{J - \mu_J}{\sigma_J}
\end{equation}

For a one-sided test, the $H0$ is rejected, if $\hat{z} > z_{1-\alpha/2}$. The implemented function can test for monotonic trend (two-sided), increasing or decreasing trend (one-sided).

\subsection{Example using \texttt{jonckheere.test}}

<<>>=
## Example from Sachs (1997, p. 402)
require(PMCMR)
x <- c(106, 114, 116, 127, 145, 110, 125,
       143, 148, 151, 136, 139, 149, 160,
       174)
g <- as.factor(c(rep(1,5), rep(2,5), rep(3,5)))
levels(g) <- c("A", "B", "C")
jonckheere.test(x , g, "increasing")
rm(x,g)
@ 

\section{Comparison of multiple joint samples (Two-factorial unreplicated complete block design)}

\subsection{Friedman test}
The linear model of a two factorial unreplicated complete block design can be written as:

\begin{equation}
  y_{i,j} = \mu +  \alpha_{i} + \pi_{j} + \epsilon_{i,j}
\end{equation}

with $\pi_{j}$ the $j$-th level of the block (e.g. the specific response of the $j$-th test person). The Friedman test is the non-parametric alternative for this type of $k$ dependent treatment groups with equal sample sizes. The null hypothesis, $H0: F(1) = F(2) = \ldots = F(k)$ is tested against the alternative hypothesis: at least one group does not belong to the same population. The response vector $y$ has to be ranked in ascending order separately for each block  $\pi_{j}: j = 1, \ldots m$. After that, the statistics of the Friedman test is calculated according to Eq. \ref{eq:friedman}:

\begin{equation}\label{eq:friedman}
\hat{\chi}^2_R = \left[ \frac{12}{n   k \left( k + 1 \right)} \sum_{i=1}^k R_i \right] - 3   n \left(k + 1\right)
\end{equation}

The Friedman statistic is approximately $\chi^{2}$-distributed and the null hypothesis is rejected, if $\hat{\chi}²_R > \chi^2_{k-1;\alpha}$.

\subsection{Friedman -- post-hoc test after Nemenyi}

Provided that the Friedman test indicates significance, the post-hoc test according to \citet{Nemenyi_1963} can be employed \citep[p. 668]{Sachs_1997}. This test requires a balanced design $\left(n_1 = n_2 = \ldots = n_k = n \right)$ for each group $k$ and a Friedman-type ranking of the data. The inequality \ref{eq:demsar} was taken from \citet[p. 11]{Demsar_2006}, where the critical difference refer to mean rank sums ($\left| \bar{R}_i - \bar{R}_j \right|$):


\begin{equation}\label{eq:demsar}
\left| \bar{R}_i - \bar{R}_j \right| > \frac{q_{\infty;k;\alpha}}{\sqrt{2}}   \sqrt{\frac{k \left( k + 1 \right)}{6   n}}
\end{equation}

 This inequality leads to the same critical differences of rank sums ($\left| R_i - R_j \right|$) when multiplied with $n$ for $\alpha = [0.1, 0.5, 0.01]$, as reported in the tables of \citet[pp. 36--38]{Wilcoxon_Wilcox_1964}. Likewise to the \texttt{posthoc.kruskal.nemenyi.test} the function \texttt{posthoc.friedman.nemenyi.test} calculates the corresponding levels of significance and the generic function \texttt{print} depicts the lower triangle of the matrix that contains these $p$-values. The test according to \citet{Nemenyi_1963} was developed to account for a family-wise error and is already a conservative test. This is the reason, why there is no $p$-adjustment included in the function. 


\subsection{Example using  \texttt{posthoc.friedman.nemenyi.test}}
This example is taken from \citet[p. 675]{Sachs_1997} and is also included in the help page of the function \texttt{posthoc.friedman.nemenyi.test}. In this experiment, six persons (block) subsequently received six different diuretics (groups) that are denoted A to F. The responses are the concentration of Na in urine measured two hours after each treatment. 

<<>>=
require(PMCMR)
y <- matrix(c(
3.88, 5.64, 5.76, 4.25, 5.91, 4.33, 30.58, 30.14, 16.92,
23.19, 26.74, 10.91, 25.24, 33.52, 25.45, 18.85, 20.45, 
26.67, 4.44, 7.94, 4.04, 4.4, 4.23, 4.36, 29.41, 30.72,
32.92, 28.23, 23.35, 12, 38.87, 33.12, 39.15, 28.06, 38.23,
26.65),nrow=6, ncol=6, 
dimnames=list(1:6,c("A","B","C","D","E","F")))
print(y)
@

Based on a visual inspection (Fig. \ref{fig:urine}), one may assume different responses of Na-concentration in urine as related to the applied diuretics.

\begin{figure}[htb]
\begin{center}
<<fig=TRUE ,echo=FALSE >>=
library("graphics")
groups <- gl(6,6,labels=colnames(y))
boxplot(as.vector(y) ~ groups)
@
\end{center}
\caption{Na-concentration (mval) in urine of six test persons after treatment with six different diuretics.}\label{fig:urine}
\end{figure}

The global test is the Friedman test, that is already implemented in the package \texttt{stats} \citep{RCoreTeam}:
<<>>=
friedman.test(y)
@

As the Friedman test indicates significance ($\chi^2(5) = 23.3, p < 0.01$), it is meaningful to conduct multiple comparisons in order to identify differences between the diuretics.

<<>>=
posthoc.friedman.nemenyi.test(y)
@

According to the Nemenyi post-hoc test for multiple joint samples, the treatment F based on the Na diuresis differs highly significant ($p < 0.01$) to A and D, and E differs significantly ($p < 0.05$) to A. Other contrasts are not significant ($p > 0.05$). This is the same test decision as given by \citep[p. 675]{Sachs_1997}. 

\subsection{Friedman -- post-hoc test after Conover}

\citet{Conover_1999} proposed a post-hoc test for pairwise comparisons, if Friedman-Test indicated significance. The absolute difference between two group rank sums are significantly different, if the following inequality is satisfied:

\begin{equation}\label{eq:fried.conover}
  \left| R_i - R_j \right| > t_{1-\alpha/2;(n-1)(k-1)} \sqrt{\frac{2 k \left(1 - \frac{\hat{\chi}^2_R}{n\left(k-1\right)}\right) \left(\sum_{i=1}^n \sum_{j=1}^k R_{i,j}^2 - \frac{n k \left(k + 1 \right)^2}{4} \right)}{\left(k-1\right)\left(n-1\right)} }
\end{equation}

Although \citet{Conover_1999} originally did not include a $p$-adjustment, the function has included the methods as given by \texttt{p.adjust}, because it is a very liberal test. So it is up to the user, to apply a $p$-adjustment or not, when using this function.

\subsection{Example using  \texttt{posthoc.friedman.conover.test}}

<<>>=
require(PMCMR)
y <- matrix(c(
3.88, 5.64, 5.76, 4.25, 5.91, 4.33, 30.58, 30.14, 16.92,
23.19, 26.74, 10.91, 25.24, 33.52, 25.45, 18.85, 20.45, 
26.67, 4.44, 7.94, 4.04, 4.4, 4.23, 4.36, 29.41, 30.72,
32.92, 28.23, 23.35, 12, 38.87, 33.12, 39.15, 28.06, 38.23,
26.65),nrow=6, ncol=6, 
dimnames=list(1:6,c("A","B","C","D","E","F")))
friedman.test(y)
posthoc.friedman.conover.test(y=y, p.adjust="none")
@

\subsection{Quade test}
Likewise to the Friedman test, the Quade test is a non-parametric test for analyzing randomized complete block designs. According to \citet{Conover_1999} the Quade test is more powerful than the Friedman test in the case of $k < 5$. 
The $H0$ is that the $k$ groups are identical, whereas $HA$ means that at least one group differs from at least one other group. First, the data are ranked within each block (i.e. row) to yield $R_{i,j}$. Then, the range in each block (maximum minus minimum value of the data) needs to be computed and ranked, $Q_i$. Then the scores are

\begin{equation}
S_{i,j} = Q_i * \left( R_{i,j} - \left(k + 1 \right) / 2 \right)
\end{equation}

and 

\begin{equation}
  S_j = \sum_{i=1}^b S_{i,j}
\end{equation}

Then the test statistic is 
\begin{equation}
  \hat{F} = \frac{\left( b - 1 \right) B}{A - B}
\end{equation}

with
\begin{eqnarray*}
  A & = & \sum_{i=1}^b \sum_{j=1}^k S_{i,j}^2 \\
  B & = & \frac{1}{b} \sum_{i=1}^k S_j^2
\end{eqnarray*}

The test statistic $\hat{F}$ is tested against the $F$-quantile for a given $\alpha$ with $df_1 = k - 1$ and $df_2 = \left(b-1\right) \left(k-1\right)$. This is equivalent to a two-way ANOVA on the scores $S_{i,j}$.

\subsection{Quade -- posthoc tests}

The package \texttt{PMCMR} offers two posthoc tests following a significant Quade test. Inequality \ref{eq:quade.mult.t} was taken from \citet{Heckert_Filliben_2003} and uses the student-t distribution.

\begin{equation}\label{eq:quade.mult.t}
  \left|S_i - S_j \right| > t_{1-\alpha/2*, \left( b-1 \right) \left( k-1 \right)} 
  \sqrt{\frac{2 b \left(A - B \right)}{\left( b-1 \right) \left( k-1 \right)} }
\end{equation}

Inequality \ref{eq:quade.mult.z} was taken from the manual of STATService 2.0 \citep{Parejo2012} and uses the standard normal distribution.

\begin{equation}\label{eq:quade.mult.z}
  \left|\frac{W_i}{n_i \left( n_i + 1 \right) / 2} - \frac{W_j}{n_j \left( n_j + 1 \right) / 2} \right| > z_{1-\alpha/2*} \sqrt{\frac{k \left( k + 1\right) \left( 2 n + 1 \right) \left( k - 1 \right)}{18 n \left( n + 1 \right)}}
\end{equation}

with
\begin{equation}
  W_j = \sum_{i=1}^b \left( Q_i * R_{i,j} \right)
\end{equation}

The calculated $p$-values can be adjusted to control Type I error inflation with the methods as given in \texttt{p.adjust}.

\subsection{Example using \texttt{posthoc.quade.test}}

<<>>=
## Conover (1999, p. 375f):
## Numbers of five brands of a new hand lotion sold in seven stores
## during one week.
y <- matrix(c( 5,  4,  7, 10, 12,
               1,  3,  1,  0,  2,
              16, 12, 22, 22, 35,
               5,  4,  3,  5,  4,
              10,  9,  7, 13, 10,
              19, 18, 28, 37, 58,
              10,  7,  6,  8,  7),
            nrow = 7, byrow = TRUE,
            dimnames =
            list(Store = as.character(1:7),
                 Brand = LETTERS[1:5]))
y
quade.test(y)
posthoc.quade.test(y, dist="TDist", p.adj="none")
@ 

\section{Comparison of multiple joint samples (Two-factorial balanced incomplete block design)}

\subsection{Durbin test}
The Durbin test can be applied in cases of an balanced incomplete block design (BIBD). The BIBD is characterized by 
\begin{itemize}
  \item Every block contains $k$ groups (experimental units)
  \item Every group appears in $r$ blocks
  \item Every group appears with every other group an equal number of times
\end{itemize}

The $H0$ is equivalent to the Friedman test or Quade test. First, rank the data $X_{i,j}$ within each block to obtain $R_{i,j}$. Then

\begin{equation}
  R_j = \sum_{i=1}^b R_{i,j}
\end{equation}

The test statistic is
\begin{equation}
  \hat{\chi}^2 = \frac{\left( t - 1 \right) \left( \sum_{j=1}^t R_j^2 - r~C \right)}{A - C}
\end{equation}

with $t$ the number of groups, $k$ the total number of groups per block, $b$ the number of blocks and $r$ the number of times each group appears.

Furthermore,

\begin{eqnarray*}
  A & = & \sum_{i=1}^b \sum_{j=1}^t R_{i,j}^2 \\
  C & = & \frac{b k * \left( k + 1 \right)^2}{4}
\end{eqnarray*}

The $H0$ is rejected if $\hat{\chi}^2 > \chi^2$ on the level of $\alpha$ with $df = t - 1$.

It should be noted that the Durbin test is equivalent to the Friedman test in case of a balanced complete block design.

\subsection{Example using \texttt{durbin.test}}

<<>>=
## Example for an incomplete block design:
## Data from Conover (1999, p. 391).
y <- matrix(c(
2,NA,NA,NA,3, NA,  3,  3,  3, NA, NA, NA,  3, NA, NA,
  1,  2, NA, NA, NA,  1,  1, NA,  1,  1,
NA, NA, NA, NA,  2, NA,  2,  1, NA, NA, NA, NA,
  3, NA,  2,  1, NA, NA, NA, NA,  3, NA,  2,  2
), ncol=7, nrow=7, byrow=FALSE,
dimnames=list(1:7, LETTERS[1:7]))
y
durbin.test(y)
@ 

\subsection{Durbin -- posthoc test}
Provided that the omnibus test of Durbin indicates, that at least one group differs from another group, multiple comparisons can be conducted according to Inequality \ref{eq:posthoc.durbin}

\begin{equation}\label{eq:posthoc.durbin}
  \| R_j - R_i \| > t_{1-\alpha/2*, bk-b-t+1} \sqrt{\frac{\left( A - C \right) 2 r}{bk - b - t + 1} \left( 1 - \frac{\hat{\chi}^2}{b * \left( k - 1 \right)} \right)}
\end{equation}

The p-values can be adjusted with \texttt{p.adjust}.

\subsection{Example using \texttt{posthoc.durbin.test}}

<<>>=
posthoc.durbin.test(y, p.adj="none")
@ 

\section{Auxiliary functions}

The package \texttt{PMCMR} comes with a \texttt{print.PMCMR} and a \texttt{summary.PMCMR} function:

<<>>=
print(posthoc.durbin.test(y, p.adj="none"))
@ 
<<>>=
summary(posthoc.durbin.test(y, p.adj="none"))
@ 

Furthermore, the function \texttt{get.pvalues} was included, to extract the p-values from an \texttt{PMCMR} object or an \texttt{pairwise.htest} object. The output of \texttt{get.pvalues} is a named numeric vector, where each element is named after the corresponding pairwise comparison. It can be further processed with \texttt{multcompLetters} from the package \texttt{multcompView} to find and indicate homogeneous groups for a given level of significance. This can be used to create plots (Fig. \ref{Fig:barplot}) or tables (Table \ref{tab:bonf.dunn}) with \texttt{xtable}.

\begin{figure}[h]
<<fig=TRUE ,echo=TRUE >>=
require(multcompView)
data(InsectSprays)
attach(InsectSprays)
out <- posthoc.kruskal.dunn.test(count ~ spray, p.adjust="bonf")
out.p <- get.pvalues(out)
out.mcV <- multcompLetters(out.p, threshold=0.05)
Rij <- rank(count)
Rj.mean <- tapply(Rij, spray, mean)
xx <- barplot(Rj.mean, ylim=c(0, 1.2* max(Rj.mean)), 
              xlab="Spray", ylab="Mean rank")
yy <- Rj.mean + 3
text(xx, yy, lab=out.mcV$Letters)
detach(InsectSprays)
@
\caption{Barplot of mean ranks of the \texttt{InsectSprays} data set. Different letters indicate significant differences ($p < 0.05$) according to the Bonferroni-Dunn test (see Chap. \ref{Dunn}).}\label{Fig:barplot}
\end{figure}

<<Dunn.tab, results=tex, echo=TRUE>>=
require(xtable)
require(multcompView)
data(InsectSprays)
attach(InsectSprays)
out <- posthoc.kruskal.dunn.test(count ~ spray, p.adjust="bonf")
out.p <- get.pvalues(out)
out.mcV <- multcompLetters(out.p, threshold=0.05)
Rij <- rank(count)
Rj.mean <- tapply(Rij, spray, mean)
dat <- data.frame(Group = names(Rj.mean),
                  meanRj = Rj.mean,
                  M = out.mcV$Letters)
dat.x <- xtable(dat)
caption(dat.x) <- "Mean ranks ($\\bar{R}_{j}$) of the 
\\texttt{InsectSprays} data set. Different letters (M) 
indicate significant differences ($p < 0.05$) according 
to the Bonferroni-Dunn test (see Chap. \\ref{Dunn})."
colnames(dat.x) <- c("Group", "$\\bar{R}_{j}$", "M")
digits(dat.x) <- 1
label(dat.x) <- "tab:bonf.dunn"
print(dat.x, include.rownames=F, caption.placement="top", 
     	      sanitize.text.function = function(x){x}, 
              table.placement="h")
detach(InsectSprays)
@

\newpage
\bibliography{PMCMR}
\end{document}


