

% setwd("d:/dev/twang/{t/doc")
% Sweave("twang.rnw"); system("texify twang.tex"); system("c:\\MiKTeX\\miktex\\bin\\yap.exe twang.dvi",wait=FALSE)

\SweaveOpts{prefix.string=twang}

\documentclass{article}
\bibliographystyle{plain}
\usepackage[active]{srcltx}
\usepackage{url}
\addtolength{\topmargin}{-0.5in}
\addtolength{\textheight}{0.75in}
\addtolength{\oddsidemargin}{-0.5in}
\addtolength{\textwidth}{1in}
\newcommand{\EV}{\mathrm{E}}
\newcommand{\Var}{\mathrm{Var}}
\newcommand{\aRule}{\begin{center} \rule{5in}{1mm} \end{center}}

 %% Tracking changes and notes code added by Dan Mc 6-30-2014 %%
\usepackage[normalem]{ulem} % for "tracked changes"
\usepackage{color} % for "tracked changes"
\newcommand{\note}[1]{{\color{blue} #1}}
\newcommand{\dele}[1]{\textcolor{red}{\sout{#1}}} % delet in an equation
\newcommand{\ins}[1]{\textcolor{blue}{\bf #1}} % insert

\title{Propensity scores for multiple treatments:\\A
tutorial for the \texttt{mnps} function in the \texttt{twang} package}

\author{Lane Burgette, Beth Ann Griffin and Dan McCaffrey 
  \footnote{The development of this software and tutorial was funded
    by National Institute of Drug Abuse grants number 1R01DA015697 (PI:
    McCaffrey) and 1R01DA034065 (PIs: Griffin/McCaffrey).} 
\\RAND Corporation}

%\VignetteIndexEntry{Propensity scores for multiple treatments: A tutorial for the mnps function}
%\VignetteDepends{gbm,survey,xtable}
%\VignetteKeywords{propensity score, nonresponse weighting}
%\VignettePackage{twang}

 \newcommand{\mathgbf}[1]{{\mbox{\boldmath$#1$\unboldmath}}}

\begin{document}

\maketitle

\section{Introduction}

The Toolkit for Weighting and Analysis of Nonequivalent Groups,
\texttt{twang}, was designed to make causal estimates in the binary
treatment setting.  In \texttt{twang} versions 1.3 and later, we have
extended this software package 
to handle more than two treatment conditions through the 
\texttt{mnps} function, which stands for
\underline{m}ulti\underline{n}omial \underline{p}ropensity
\underline{s}cores.  McCaffrey et al.\ (2013) describe the 
methodology behind the \texttt{mnps} function; the purpose of this
document is to describe the syntax and features related to the
implementation in \texttt{twang}.  

At a high level, the \texttt{mnps} function decomposes
the propensity score estimation into several applications of the \texttt{ps}
function, which was designed for the standard dichotomous treatment
setting.  For this 
reason, users who are new to \texttt{twang} are encouraged to learn
about the \texttt{ps} function before using the \texttt{mnps}
function.  The other vignette that accompanies the package (Ridgeway et
al., 2014) provides an extensive overview of the \texttt{ps} function,
and much of that information will not be repeated here.  


\section{An ATE example}

<<echo=FALSE>>=
options(width=80)
@



To demonstrate the package we use a random subset of the data
described in McCaffrey et al.\ (2013). This truncated dataset is called
\texttt{AOD}, and is included in the package.  There are three
treatment groups in the study, and the data include records for 200 youths
in each treatment group of an alcohol and other drug treatment
evaluation.  We begin by loading the 
package and the data.\footnote{Code used in this tutorial can be found in
stand alone text file at \url{http://www.rand.org/statistics/twang/downloads.html/mnps_turorial_code.r.}}   
%Because there is a stochastic component to the
%subsequent model fits, we also set the random seed to ensure full
%replicability.
%\note{DFM: Is there really a stochastic component anymore
%by default the bag fraction is 1 so there is no stochastics component.
%I removed this from the SAS tutorial and don't recall if it is still in 
%the R ps tutorial}

<<>>=
library(twang)
data(AOD)
set.seed(1)
@

For the \texttt{AOD} dataset, the variable \texttt{treat} contains the
treatment indicators, which have possible values \texttt{community},
\texttt{metcbt5}, and \texttt{scy}.  The other variables included in
the dataset are:
\begin{itemize}
\item \texttt{suf12}: outcome variable, substance use frequency at 12
  month follow-up
\item \texttt{illact}: pretreatment covariate, illicit activities scale
\item \texttt{crimjust}: pretreatment covariate, criminal justice involvement
\item \texttt{subprob}: pretreatment covariate, substance use problem scale
\item \texttt{subdep}: pretreatment covariate, substance use dependence scale
\item \texttt{white}: pretreatment covariate, indicator for non-Hispanic white youth
\end{itemize}

In such an observational study, there
are several quantities that one may be interested in estimating.  
The estimands that are most commonly of interest are the average
treatment effect on the population (ATE) and the average treatment
effect on the treated (ATT).  The differences between these
quantities are explained at length in McCaffrey et al.\ (2013), but in
brief the ATE answers the question of
how, on average, 
the outcome of interest would change if everyone in the population of
interest had been assigned to a particular treatment relative to if
they had all received another single treatment.  The ATT answers the
question of how the average outcome would 
change if everyone who received one particular treatment had instead received
another treatment. We first demonstrate the use of \texttt{mnps} when ATE
is the effect of interest and then turn to using the function to support 
estimation of ATT.

\subsection{Estimating the weights}

The main argument for the \texttt{mnps} function is a formula with the
treatment variable on the left-hand side of a tilde, and pretreatment variables
on the right-hand side, separated by plus signs.  Other key arguments
are \texttt{data}, which 
simply tells the function the name of the dataframe that contains the
variables for the propensity score estimation; the \texttt{estimand},
which can either be ``ATT'' or ``ATE''; and \texttt{verbose}, which if
set as \texttt{TRUE} instructs the function to print updates on the
model fitting process, which can take a few minutes.  

<<>>=
mnps.AOD <- mnps(treat ~ illact + crimjust + subprob + subdep + white,
                 data = AOD, 
                 estimand = "ATE", 
                 verbose = FALSE, 
                 stop.method = c("es.mean", "ks.mean"), 
                 n.trees = 3000)
@ 

The \texttt{twang} methods rely on tree-based regression models that
are built in an iterative fashion.  As the iterations or number of
regression trees added to the model increases, the
model becomes more complex.  However, at some point, 
more complex models typically result in worse balance on the
pretreatment variables and therefore are less useful in a propensity
score weighting context.  The
\texttt{n.trees} argument controls the 
maximum number of iterations.

Another key choice is the measure of balance that one uses when
fitting these models.  This is specified in the \texttt{stop.method}
argument.  As with the \texttt{ps} function, four \texttt{stop.method}
objects are included in the package.  They are 
\texttt{es.mean}, \texttt{es.max}, \texttt{ks.mean}, and
\texttt{ks.max}.   
The four stopping rules are defined by two components:
a balance metric for covariates and rule for summarizing across
covariates.  The balance metric summarizes the difference between two
univariate distributions of a single pretreatment variable (e.g.,
illicit activities scale).  
The default stopping rules in \texttt{twang} use two balance
metrics: absolute standardized mean difference (ASMD; also referred to as the absolute
standardized bias or the effect size (ES)) and
the Kolmogorov-Smirnov (KS) statistic.  The stopping rule use two
different rules for summarizing across covariates: the mean of the
covariate balance metrics (``mean'') or the maximum of the balance
metrics (``max'').  The first piece of the stopping rule name
identifies the balance metric (ES or KS) and the second piece
specifies the method for summarizing across balance metrics.  For
instance, \texttt{es.mean} uses the effect
size or ASMD and summarizes across variables
with the mean and the \texttt{ks.max} uses the KS statistics to assess
balances and summarizes using the maximum across variables and the
other two stopping rules use the remaining two combinations of balance
metrics and summary statistics.  In this example, we chose to examine
both \texttt{es.mean} and \texttt{ks.mean}.  

After running the \texttt{mnps()} command, a useful first step is to
make sure that we let the models run for a sufficiently 
large number of iterations in order to optimize the balance
statistics of interest.  We do this by seeing whether any of the balance 
measures of interest still appear to be decreasing after the number of
iterations specified by the argument \texttt{n.trees} which we set 
to 3,000 for this example (10,000 iterations
is the default).

\begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 7, width = 10.5>>= 
    plot(mnps.AOD, plots = 1)
@
\end{center}

As noted above, \texttt{mnps} estimates weights by repeated use of
the \texttt{ps} function and comparing each treatment the pooled sample 
of other treatments.  The figure has one plot corresponding to each 
of those fits. Each plot is then further divided into one panel for each
stopping rule used in the estimation. Since we used the ``es.mean'' and
``ks.mean'' stopping rules there are two panels in each plot.
By default the plots for the different treatments are plotted in 
a single row; setting the height and width of the graphics device
can make the plots easier to view.
In this figure, it appears that each of the balance measures are
optimized with substantially fewer than 3,000 iterations, so we do not have
evidence that we should re-run the \texttt{mnps()} call with a higher
number of iterations or trees.

A key assumption in propensity score analyses is that each
experimental unit has a non-zero probability of receiving each
treatment.  The plausibility of this assumption may be assessed by
examining the 
overlap of the empirical propensity score distributions.  This
diagnostic is available using 
the \texttt{plots = 2} argument in the \texttt{plot} function.  
We use the \texttt{subset} option to specify which stopping rule 
we wish present in the plot.\footnote{The value for the \texttt{subset} argument 
can be a character variable with the name of the stopping, as
was used in the example code, or a number corresponding to
the stopping rule. Stopping rules are numbered by the alphabetical
ordering among the rules specified in the \texttt{mnps} call.}


\pagebreak 

  \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 8, width = 7>>= 
    plot(mnps.AOD, plots = 2, subset = "es.mean")
@

\end{center}

Here, the overlap assumption generally seems to be met, although there
should be some concern that adolescents in the metcbt5 and scy
conditions do not overlap well with the community group given the top
most graphic. See McCaffrey et al.\ (2013) for more details on this
issue. 

\subsection{Graphical assessments of balance}

As with the \texttt{ps} function for the binary treatment setting, the default
plotting function for \texttt{mnps}-class objects also displays information
on commonly-used balance statistics.  
In particular, when the \texttt{plots} argument is set
equal to 3, it provides comparisons of the absolute standardized mean
differences (ASMD) between the treatment groups on the pretreatment
covariates, before and after weighting. When the \texttt{plots}
argument is set equal to 4, the display is of $t$- and chi-squared
statistic $p$-values comparing the two groups before and after
weighting. However, 
whereas there is a single plot for these   
balance diagnostics in the binary treatment setting, in the multiple
treatment case, one can either examine a plot for each of the
treatment conditions, or summarize the balance statistics in some way,
across the treatment conditions.
As a default, the \texttt{plot} function for an \texttt{mnps} object
returns the maximum of the pairwise balance statistics across treatment
groups for each of the covariates:

<<echo=FALSE>>=
options(width=120)
@

  \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 3.5, width = 5>>= 
    plot(mnps.AOD, plots = 3)
@
\end{center}

As shown here, after weighting, the maximum ASMD
decreases for all pretreatment covariates.  The statistically
significant difference (before 
taking the maximum across treatment groups) is indicated by the solid
circle. One may see the balance plots for the  
individual fits by setting the \texttt{pairwiseMax} argument to
\texttt{FALSE}.  

<<echo=FALSE>>=
options(width=120)
@


\pagebreak

 \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 11, width = 8>>= 
plot(mnps.AOD, plots = 3, pairwiseMax = FALSE, figureRows = 3)
@
\end{center}

%\note{Maybe I am being too simplistic but I couldn't understand
%how the max of the pairwise std effect in the individual 
%pairwise plots was about 0.10 but the max in maximum plot was
%greater than 0.20.  What am I missing?  We need to explain this
%in the text.  Using twang 1.4, the y-axis label on the max plot
%did not include "(maximum pairwise)". Also, the pairwise plots
%had side by side panels for the stopping rules and they had
%a better aspect ratio.}
The additional \texttt{figureRows} argument instructs the function to
spread the plots over three rows; by default the plots would be
arranged in a single row rather than a column.  We note here that red
lines represent pretreatment covariates for which the pairwise ASMDs
increase after weighting.

Setting the \texttt{plots} argument equal to 4 displays $t$-test or
$\chi^2$ statistic pairwise minimum $p$-values for differences 
between each of the individual treatment groups and observations in
all other treatment groups.  

\pagebreak

\begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 4.5, width = 8>>= 
    plot(mnps.AOD, plots = 4)
@

\end{center}

<<echo=FALSE>>=
options(width=80)
@

As seen in this figure, the pairwise minimum $p$-values all increase
after propensity score weighting.

Some of the figures include many frames, which can result in figures
that are too big or difficult to read for some methods of display. To
control this, three controls are available. First, the
\texttt{treatments} argument can be used to specify only comparisons
that involve a specific treatment level or, in the ATE case, only
comparisons between two specified treatment levels. Similarly, the
\texttt{singlePlot} argument . For example, \texttt{singlePlot = 2}
would display only the second frame of those produced by the plot
command (see figure below). Finally, specifying \texttt{multiPage = TRUE} prints the
frames in succession.  If this option is used after specifying a file
to plot to (e.g., using \texttt{pdf()}), the frames will be printed on
separate pages.

  \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 5, width = 7>>= 
    plot(mnps.AOD, plots = 2, subset = "es.mean", singlePlot = 2)
@

\end{center}

\subsection{Tabular assessments of balance}

Beyond graphics, there are several other functions that may be of
interest to \texttt{mnps} users.  The first is given by the 
\texttt{bal.table} function.  For propensity score analyses with
multiple treatments, this function returns a lot of information. The
intention with this function is that its output be loaded into a
spreadsheet software program.  (E.g., one can write the output into a .csv
file using the \texttt{write.csv} function and open the resulting file
using a spreadsheet application.)  For
each outcome category, and each stopping rule (in addition to the
unweighted analysis) the \texttt{bal.table} function gives balance
statistics such as weighted and unweighted means by treatment group.


<<>>=
bal.table(mnps.AOD, digits = 2)
@ 

%\note{DFM: would it be useful to show that we can save the
%results of bal.table?  Also, there if you use digits it only 
%save 2 digits.}
As of version 1.4 of TWANG, the balance measures are given for all
pairwise combinations.  (Prior to that version the balance measures
were reported for each treatment against all others; we feel that the
pairwise comparisons give a fuller accounting of balance in ATE
applications.) 


More parsimonious versions of the summaries are available using the
\texttt{collapse.to} argument.  Setting \texttt{collapse.to = 'covariate'} 
gives the maximum of the ASMD and the minimum of the $p$-value across
all pairwise comparisons for each
pretreatment covariate and stopping rule.  


<<>>=
bal.table(mnps.AOD, collapse.to = 'covariate', digits = 4)
@ 


As shown, for each pretreatment variable, the maximum ASMD has
decreased and the minimum $p$-values have increased after applying
weights that arise from either stop.method.

Another useful summary table sets \texttt{collapse.to = 'stop.method'} 
which further collapses the results above so that we summarize balance
across all covariates and all pairwise group comparisons.  

<<>>=
bal.table(mnps.AOD, collapse.to = 'stop.method', digits = 4)
@ 

Here we quickly see how the maximum ASMDs and minimum $p$-values have
all moved in the desired direction after propensity score
weighting. 

Rather than collapsing the values of the table as described
above, there are also several options for subsetting the \texttt{bal.table}
output. The arguments
\texttt{subset.var} and \texttt{subset.stop.method} instruct the
function to include only the 
covariates indicated, and stop.method results indicated, respectively. The
\texttt{subset.treat} instructs the function to return only the
pairwise comparisons including the specified treatment or, if two
treatment levels are indicated, the pair-wise comparisons that include
those two treatments.  Note that \texttt{subset.treat} may not be
used when \texttt{collapse.to} is specified as \texttt{'stop.method'}
or \texttt{'covariate'}. Further, the table may be subset on the basis
of ES and KS and the related $p$-values via the \texttt{es.cutoff},
\texttt{ks.cutoff}, \texttt{p.cutoff}, and \texttt{ks.p.cutoff}
arguments.  These cutoffs exclude rows that are well-balanced as
measured by the corresponding . For example \texttt{p.cutoff = 0.1}
would exclude rows with $p$-values greater than 10\%, and
\texttt{es.cutoff = 0.2} excludes rows with ES values below 0.2 in
absolute value. Examples of the use of these
subsetting arguments are given below.

<<>>=
bal.table(mnps.AOD, subset.treat = c('community', 'metcbt5'), 
          subset.var = c('white', 'illact', 'crimjust'))
@ 

<<>>=
bal.table(mnps.AOD, subset.stop.method = 'es.mean', collapse.to = 'covariate')
@ 

<<>>= 
bal.table(mnps.AOD, es.cutoff = 0.1)
@ 


Finally, there is also \texttt{summary} method for the \texttt{mnps}
objects which gives the collapsed version of \texttt{bal.table()} as
well as information about the effective sample sizes for each
treatment group under each stop.method. The \texttt{summary} function 
for an mnps output object does not have a \texttt{digits} argument.

<<>>=
summary(mnps.AOD)
@ 


<<echo=FALSE>>=
options(width=80)
@

After examining the graphical and tabular diagnostics provided by
\texttt{twang}, we can analyze the outcome variable using the
propensity scores generated by the \texttt{mnps} function.  Although
two stop methods were specified initially (\texttt{es.mean} and
\texttt{ks.mean}), at this point we have to commit to a single
set of weights.  From the \texttt{bal.table} call above, we see that the
balance properties are very similar for the two stopping rules, and
from the \texttt{summary} statement, we see that the effective sample
sizes (ESS) are similar as well.  Hence, we expect the
two stop methods to give similar results; we choose to analyze the
data with the \texttt{es.mean} weights. 

\subsection{Estimating treatment effects}

In order to analyze the data using the weights, it is recommended that
one use the \texttt{survey} package, which performs weighted
analyses.  We can add the weights to the dataset using the
\texttt{get.weights} function and specify the
survey design as follows:

<<>>=
library(survey)
AOD$w <- get.weights(mnps.AOD, stop.method = "es.mean")
design.mnps <- svydesign(ids=~1, weights=~w, data=AOD)
@ 

As shown in the \texttt{ps} vignette, we can then perform the
propensity score-adjusted regression using the \texttt{svyglm} function:
<<>>=
glm1 <- svyglm(suf12 ~ as.factor(treat), design = design.mnps)
summary(glm1)
@ 

By default, \texttt{svyglm} includes dummy variables for MET/CBT-5 and
SCY, Community is the holdout group (the holdout is the group with the label that 
comes first alphabetically). Consequently, the estimated effect for
MET/CBT-5 equals  
the weighted mean for the MET/CBT-5 sample less the weighted mean for 
the Community sample, where both means are weighted to match the overall 
sample. Similarly, the effect fro SCY equals the difference in the
weighted means for the SCY and Community samples. The coefficients 
estimate the causal effects of MET/CBT-5 vs.\ Community and SCY vs.\ Community, 
respectively, assuming there are no unobserved confounders.  Using this small 
subset of the data, we are unable to detect differences in the treatment group
means. In the context of this application, the signs of the estimates 
correspond to higher substance use frequency for youths exposed to MET/CBT-5 
or SCY relative to Community.  More details on how to obtain
all relevant pairwise differences can be found in McCaffrey et al.\
(2013).  

As an alternative to estimating the pairwise differences,
we could also estimate the causal effect of each treatment relative 
to the average potential outcome of all the treatments. This estimate
is easy to obtain using \texttt{svyglm} through the use of the
\texttt{constrast} argument in the function.
<<>>=
glm2 <- svyglm(suf12 ~ treat, design = design.mnps, contrast=list(treat=contr.sum))
summary(glm2)
@

The function now provides the estimates for Community and MET/CBT-5. 
It labels them ``treat1'' and ``treat2'' because it uses their numeric
codings rather than the factor levels. We have seen previously that the
factor levels for treatment are ``community'', ``metcbt5'', and ``scy''
as levels, 1, 2, and 3. Relative to the average of all the treatments, the 
weighted Community group has lower substance use and the weighted MET/CBT-5 
group has higher use. The SCY estimate is not reported because it is a 
linear combination of the other to estimates. It can be found by: 
<<>>=
-sum(coef(glm2)[-1])
@
The standard error of this estimate can be calculated using the 
covariance matrix for the estimated coefficients:
<<>>= 
sqrt(c(-1,-1) %*% summary(glm2)$cov.scaled[-1,-1] %*% c(-1,-1))
@
The SCY mean is about equal to the average and the difference between 
them is very small relative to its standard error.

%\end{document}


\section{An ATT example}

\subsection{Estimating the weights}

It is also possible to explore treatment effects on the treated (ATTs)
using the \texttt{mnps} function.  A key difference in the
multiple treatment setting is that we must be clear as to which treatment
condition ``the treated'' refers to.  This is done through the
\texttt{treatATT} argument. Here, we define the treatment group of
interest to be the community group; thus, we are trying to draw
inferences about the relative effectiveness of the three treatment
groups for individuals like those who were enrolled in the community
program. 

<<>>=
mnps.AOD.ATT <- mnps(treat ~ illact + crimjust + subprob + subdep + white,
                     data = AOD, 
                     estimand = "ATT", 
                     treatATT = "community", 
                     verbose = FALSE, 
                     n.trees = 3000, 
                     stop.method = c("es.mean", "ks.mean"))
@ 

\subsection{Graphical assessments of balance}

The same basic graphical descriptions are available as in the ATE
case, though it is important to note that these comparisons all assess
balance relative to the ``treatment'' group rather than by comparing
balance for all possibly pairwise treatment group comparisons as is
done with ATE.  

  \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 4.5, width = 10.5>>= 
    plot(mnps.AOD.ATT, plots = 1)
@
\end{center}

\pagebreak

 \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 4.5, width = 10.5>>= 
    plot(mnps.AOD.ATT, plots = 3)
@
\end{center}

 \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 4.5, width = 10.5>>= 
    plot(mnps.AOD.ATT, plots = 3, pairwiseMax = FALSE)
@
\end{center}

%\end{document}


 \begin{center}
<<fig=TRUE, echo=TRUE, include = TRUE, height = 4.5, width = 10.5>>= 
    plot(mnps.AOD.ATT, plots = 4)
@
\end{center}

<<echo=FALSE>>=
options(width=85)
@


\subsection{Tabular assessments of balance}

The \texttt{bal.table} output is similar to the ATE case.  However,
for ATT, we only report pairwise comparisons that include the
\texttt{treatATT} category.   

<<>>=
bal.table(mnps.AOD.ATT, digits = 2)
@ 

<<>>=
bal.table(mnps.AOD.ATT, digits = 2, collapse.to  = "covariate")
@ 

<<>>=
bal.table(mnps.AOD.ATT, digits = 3, collapse.to = "stop.method")
@ 


<<echo=FALSE>>=
options(width=80)
@

\subsection{Estimating treatment effects}

The process to analyze the outcome variable is also similar: 

<<>>=
require(survey)
AOD$w.ATT <- get.weights(mnps.AOD.ATT, stop.method = "es.mean")
design.mnps.ATT <- svydesign(ids=~1, weights=~w.ATT, data=AOD)
@ 


<<>>=
glm1 <- svyglm(suf12 ~ as.factor(treat), design = design.mnps.ATT)
summary(glm1)
@ 

Note in this case that the estimated treatment effect of community
on those
exposed to the community treatment is slightly stronger than in the
ATE case (high numbers are bad for the outcome variable).  Although
not statistically significant, such differences 
are compatible with the notion that the youths who actually received the
community treatment responded more favorably to it than the
``average'' youth would have
(where the average is taken across the whole collection of youths
enrolled in the study).  

The discussion in McCaffrey et al.\ (2013) may be useful for
determining whether the ATE or ATT is of greater interest in a
particular application.  


\section{Conclusion}

Often, more than two treatments are available to study participants.  If the
study is not randomized, analysts may be interested in using a
propensity score approach.  Previously, few tools existed to aide the
analysis of such data, perhaps tempting analysts to ignore all but two
of the treatment conditions.  We hope that this extension to the
\texttt{twang} package will encourage more appropriate analyses of
observational data with more than two treatment conditions.  

\section*{Acknowledgements}

The random subset of data was supported by the Center for Substance
Abuse Treatment (CSAT), Substance Abuse and Mental Health Services
Administration (SAMHSA) contract \# 270-07-0191 using data provided by
the following grantees: Adolescent Treatment Model (Study: ATM:
CSAT/SAMHSA contracts \# 270-98-7047, \# 270-97-7011, \#2 77-00-6500,
\# 270-2003-00006 and grantees: TI-11894, TI-11874, TI-11892), the
Effective Adolescent Treatment (Study: EAT; CSAT/SAMHSA contract
\# 270-2003-00006 and grantees: TI-15413, TI-15433, TI-15447,
TI-15461,  TI-15467, TI-15475, TI-15478, TI-15479, TI-15481, TI-15483,
TI-15486, TI-15511, TI15514, TI-15545, TI-15562, TI-15670, TI-15671,
TI-15672, TI-15674, TI-15678, TI-15682, TI-15686, TI-15415, TI-15421,
TI-15438, TI-15446, TI-15458, TI-15466, TI-15469, TI-15485, TI-15489,
TI-15524, TI-15527, TI-15577, TI-15584, TI-15586, TI-15677), and the
Strengthening Communities-Youth (Study: SCY; CSAT/SAMHSA contracts
\# 277-00-6500, \# 270-2003-00006 and grantees: TI-13305, TI-13308,
TI-13313, TI-13322, TI-13323, TI-13344, TI-13345, TI-13354).  The
authors thank these grantees and their participants for agreeing to
share their data to support the development of the \texttt{mnps} functionality.


\begin{thebibliography}{77}     % start the bibliography

\small                          % put the bibliography in a small font


\bibitem{psChapt} Burgette, L.F., D.F. McCaffrey, B.A.\ Griffin
  (forthcoming). ``Propensity score estimation with boosted
  regression.'' In W.\ Pan and H.\ Bai (Eds.) {\em Propensity Score
    Analysis: Fundamentals, Developments and Extensions}.  New York:
    Guilford Publications, Inc.  

\bibitem{mccaf} McCaffrey, D.F., B.A. Griffin, D.\ Almirall, M.E.\
  Slaughter, R.\ Ramchand, and L.F.\ Burgette (2013).  ``A tutorial on
  propensity score estimation for multiple treatments using
  generalized boosted models.'' Forthcoming at {\em Statistics in Medicine}.

\bibitem{psVig} Ridgeway, G., D.\ McCaffrey, B.A.\ Griffin, and L.\
  Burgette (2014). ``twang: Toolkit for weighting and analysis of
  non-equivalent groups.'' Available at
  http://cran.r-project.org/web/packages/twang/vignettes/twang.pdf. 

\end{thebibliography}           % end the bibliography




\end{document}


\texttt{means.table}
which provides a simple summary of balance across the groups.
When \texttt{estimand} is set as \texttt{'ATE'}, the table shows the
population means for each pretreatment covariate in the first column
as well as each treatment group's unweighted and ATE weighted means
and corresponding unweighted and weighted population
standardized mean differences. As shown in the table below,
incorporation of the ATE propensity score weights improves each
treatment groups overall balance with the population means for each
pretreatment covariate. The function also includes an argument called
\texttt{includeSD} whose default is \texttt{FALSE}; changing it to 
\texttt{TRUE} returns standard deviations for each of the treatment
conditions (not shown). 

<<echo=FALSE>>=
options(width=85)
@

<<>>=
means.table(mnps.AOD, stop.method = "es.mean", digits = 3)
@ 


More extensive balance information is 

Although the same basic graphical descriptions are available as in the
ATE case, note that the population means above are replaced with the
means of the \texttt{treatATT} category in the \texttt{means.table}
call.  

<<>>=
means.table(mnps.AOD.ATT, digits = 3)
@ 

