Tests for vector hypotheses $H_0: \theta ={\theta }_0$ against $H_1:\theta \ne {\theta }_0$ in ${ℝ}^k$  typically have powers depending on quadratic forms of type$\lambda =\left(\theta -{\theta }_0\right)\text{'}{\Xi }^{-1}\left(\theta -{\theta }_0\right)$ . This study examines the case that $\left(\mu -{\mu }_0\right)$ is restricted to subspaces, for example, $\left(\mu -{\mu }_0\right)\text{'}=c\left(1,1,0,\dots ,0\right)$ differing only in their first two coordinates. These are called directed alternatives. The spectral decomposition of $\Xi$  supports the identification of one–dimensional alternatives least likely and most likely to be discerned, to complement conventional data analysis. Applications are drawn in the use of Hotelling’s ${{\rm T}}^2$ and of $F$ –tests in linear inference. Moreover, it is seen that a given design may be recast so as to reverse the least likely and most likely alternatives. Numerical examples serve to illustrate the findings.

$62J05,62H10and62P30$

Keywords: Linear models; $F$ tests; Hotelling's $T^2$ tests; Directed alternatives; Reversal designs

Power in statistical inference is driven by non–null distributions. For observations in ${ℝ}^k$ having dispersion matrix$\Xi$ , noncentrality parameters often emerge as the Mahalanobis [1] distance between points $\left(u,v\right)$ in${ℝ}^k$ , namely,

$D_{\Xi }^2\left(u,v\right)=\left(u-v\right)\text{'}{\Xi }^{-1}\left(u-v\right).$              (1)

This specializes to the Euclidean metric for the case that $\Xi =c{{\rm I}}_k$ , in which case the model is called isotropic. In particular, nonparametric and other statistics often have noncentral chi–squared distributions, either in small samples or asymptotically. In addition, pervasive venues in parametric inference, to be reexamined in some detail, include the following.

Case (i). Hotelling [2] Test: $T^2=n\left(\overline{Y}-{\mu }_0\right)S^{-1}\left(\overline{Y}-{\mu }_0\right)$  where $\left(\overline{Y},S\right)$ are the sample mean and dispersion matrix of $n$ Gaussian vectors in ${ℝ}^k$ having the location–scale parameters $(\mu ,\Sigma )$ . Then in testing $H_0:\mu ={\mu }_0$  against $H_1:\mu \ne {\mu }_0$ , the power function is $\Psi \left(\lambda \right)$ with noncentrality $\lambda =n\left(\mu -{\mu }_0\right)\text{'}{\Sigma }^{-1}\left(\mu -{\mu }_0\right).$

Case (ii). The General Linear Model: $\left\{Y=X\beta +\epsilon \right\}$ with Gaussian errors having zero means and dispersion matrix ${\sigma }^2{I}_n$ . Then in testing $H_0:\beta ={\beta }_o$  against $H_1:\beta \ne {\beta }_o$  in ${ℝ}^k$ , the power function is $\Psi \left(\lambda \right)$ with noncentrality $\lambda =\left(\beta -{\beta }_o\right)\text{'}X\text{'}X\left(\beta -{\beta }_o\right)/{\sigma }^2$ .

Classical theory allows for any $\left(\mu -{\mu }_0\right)\in {ℝ}^k$  for Case (i), and $\left(\beta -{\beta }_o\right)\in {ℝ}^k$  for Case (ii). On the other hand, alternatives lying in designated subspaces may hold substantive interest per se. For example, taking $\left(\mu -{\mu }_0\right)\text{'}=c\left(1,1,0,\dots ,0\right)$ allows for discrepancies between $\mu$ and ${\mu }_0$ in their first two coordinates only, whereas $\left(\mu -{\mu }_0\right)\text{'}=c\left(1,1,\dots ,1\right)$ allows for deviations along the equiangular line in ${ℝ}^k$ . Both are one–dimensional; subspaces of dimension greater than one are considered subsequently. Alternatives lying in designated subspaces of ${ℝ}^k$ are called directed alternatives, and the goal here is to study powers of tests against alternatives of these types.

The present study expands on this as follows. Not only do distinct alternatives differ in importance to users, but so too their probabilities of detection. Here the spectral decomposition of $\Xi$ , if anisotropic, supports the identification of alternatives least likely and most likely to be discovered, as well as intermediate cases. These serve to bracket the effective range of inferences intrinsic to a given study, and thereby complement conventional options in data analysis. Applications are drawn in the use of Hotelling’s $T^2$ in multivariate samples, and of F–tests in the analysis of linear models. Moreover, it is shown that a given design may be modified so as to reverse the least likely and most likely alternatives, in the event that this would better serve the objectives of an experiment.

This study is organized as follows. Supporting developments are given next in Section 2, followed by the principal findings of Section 3. Several examples in Section 4 illustrate the essential results. Collateral materials are deferred for completeness to an Appendix.

Notation

Spaces include ${ℝ}^n$ as Euclidean $n$ -space; ${ℝ}_{{}^{+}}^n$ as its positive orthant; ${\mathbb{S}}_n$  as the real symmetric $\left(n\times n\right)$ matrices; ${\mathbb{S}}_n^{+}$ as their positive definite varieties; ${\mathbb{F}}_{n\times k}$ as the real $(n\times k)$ matrices of rank $k\le n$ ; and ${\mathcal{O}}_k$ as the $(k\times k)$  orthogonal group. Vectors and matrices are set in bold type; the transpose, inverse, trace, and determinant of $A$ are $A\text{'}$ , $A^{-1}$ , $tr\left(A\right)$ , and $\left|A\right|$ ; the unit vector in ${ℝ}^n$ is $1_n=\left[1,\dots ,1\right]\text{'}$ ; $I_n$ is the $\left(n\times n\right)$ identity; and $Diag\left(A_1,\dots ,A_k\right)$ is a block-diagonal array. If $B=\left[b_1,\dots ,b_k\right]$ is of order $\left(n\times k\right)$ and rank $k<n$ , then $S_p\left(B\right)$ designates the column span of $B$ , i.e., the $k$ –dimensional subspace of ${ℝ}^n$ spanned by $\left[b_1,\dots ,b_k\right]$ . The ordered eigenvalues of $A\in {\mathbb{S}}_n$ are $\left\{{\lambda }_i\left(A\right)={\alpha }_i;1\le i\le n\right\}$ with $\left\{{\alpha }_1\ge {\alpha }_2\ge \dots \ge {\alpha }_n\right\}$ , and its spectral decomposition is $A=P{D}_{\alpha }P\text{'}={\Sigma }_{i=1}^n{\alpha }_i{p}_i{p}_i\text{'}$ , where $P=\left[p_1,\dots ,p_n\right]\in {\mathcal{O}}_n$ and $D_{\alpha }=Diag\left({\alpha }_1,\mathrm{...},{\alpha }_n\right)$ . By convention its condition number is $C_1\left(A\right)={\alpha }_1/{\alpha }_n$ . The singular decomposition of $B\in {\mathbb{F}}_{n\times k}$ is $B=P{D}_{\delta }Q\text{'}={\Sigma }_{i=1}^k{\delta }_i{p}_i{q}_i\text{'}$ , where the mutually orthogonal columns of $P=\left[p_1,\dots ,p_k\right]$ comprise the left–singular vectors; $D_{\delta }=Diag\left({\delta }_1,\dots ,{\delta }_k\right)$ are its singular values; and columns of $Q\in {\mathcal{O}}_k$ are the right–singular vectors.

Special Distributions

For $Y\in {ℝ}^n$ , its distribution, mean, and dispersion matrix are L(Y), $E\left(Y\right)=\mu$  and $V\left(Y\right)=\Sigma$ , say, with variance $Var\left(Y\right)={\sigma }^2$ on ${ℝ}^1$ . Specifically, L(Y) $=N_n\left(\mu ,\Sigma \right)$ is Gaussian on ${ℝ}^n$ with parameters $\left(\mu ,\Sigma \right)$ . Distributions on ${ℝ}_{{}^{+}}^1$ include the $\chi 2(\cdot ;\nu ,\lambda )$ with $\nu$ degrees of freedom and noncentrality parameter $\lambda$ ; the Snedecor–Fisher $F(\cdot ;{\nu }_1,{\nu }_2,\lambda )$ with degrees of freedom $\left({\nu }_1,{\nu }_2\right)$ and noncentrality $\lambda$ ; and Hotelling [2] $T_k^2(\cdot ,\nu ,\lambda )$ of order $k$ having $\nu$ degrees of freedom and noncentrality $\lambda$ . Recall that $F(\cdot ;{\nu }_1,{\nu }_2,\lambda )$ increases stochastically with increasing $\lambda$ with other parameters held fixed. Identify $c_{\alpha }$ in context as the upper $\alpha$ –level rejection rule. The power of a test, to be considered as a function of $\lambda$ , is designated by $\psi (\lambda )$ .

Directed alternatives

Our notation encompasses both (i) Hotelling [2] $T^2$ and (ii) General Linear Models, having location–scale parameters $\left(\delta ,\Xi \right)$ . What distinguishes this study are directed alternatives with examples as noted, but expanded to include alternatives $\left\{{\theta }_j;1\le j\le k\right\}$ aligned with the orthonormal eigenvectors $Q=\left[q_1,\dots ,q_k\right]$ of $\Xi$ , thus standardized to unit lengths. To continue, as $\Omega ={\Xi }^{-1}$ assumes a central role, take $\Omega ={\Sigma }_{i=1}^k{\kappa }_i{q}_i{q}_i\text{'}=Q{D}_{\kappa }Q\text{'}\in {\mathbb{S}}_k^{+}$ as its spectral decomposition, with $\left\{{\kappa }_1\ge \dots \ge {\kappa }_k>0\right\}$ . As in Appendix A.1, undertake the expansions

$\Omega = Q_1{D}_{{\kappa }_1}{Q}_1\text{'}+Q_2{D}_{{\kappa }_2}{Q}_2\text{'}+Q_3{D}_{\kappa }{}_{{}_3}{Q}_3\text{'}$ (2)

$\Omega = Q_1{D}_{{\kappa }_1}{Q}_1\text{'}+k_r{Q}_2{D}_{{\kappa }_2}{Q}_2\text{'}+Q_3{D}_{\kappa }{}_{{}_3}{Q}_3\text{'};k_r repeateds times$ ,          (3)

where elements of $Q\text{=[}{Q}_{\text{1}}\text{,}{Q}_{\text{2}}\text{,}{Q}_{\text{3}}\text{]}$ are of orders $\left\{\left(k\times \left(r-1\right)\right),\left(k\times s\right),\left(k\times d\right)\right\}$ with $d=k-r-s+1$ , and where $D_k=Diag\left(D_{\kappa }{}_{{}_1},D_{\kappa }{}_{{}_2},D_{\kappa }{}_{{}_3}\right)$ is partitioned conformably. In regard to quadratic forms of type $Q\left(u\right)=u\text{'}\Omega u$ serving as noncentrality parameters, a principal result is the following.

Theorem 1. Given is a location–scale model with parameters $\left(\delta ,\Xi \right)$ , together with a test for $H_0:\delta ={\delta }_0$ against $H_1:\delta \ne {\delta }_0$ having power $\Psi \left(\lambda \right)$ increasing monotonically with $\lambda =D_{\Xi }^2\left(\delta ,{\delta }_0\right)=\left(\delta -{\delta }_0\right)\text{'}{\Xi }^{-1}\left(\delta -{\delta }_0\right)$ . Take $\left\{\left(\delta -{\delta }_0\right)={\theta }_j;1\le j\le k\right\}$ in succession as the eigenvectors $\left\{{\theta }_j=q_j;1\le j\le k\right\}$ of ${\Xi }^{-1}$ with eigen values $\left\{{\kappa }_1\ge \dots \ge {\kappa }_k>0\right\}$ .

1. Then powers $\Psi \left({\lambda }_j\right)$ of the test at alternatives $\left\{\left(\delta -{\delta }_0\right)={\theta }_j;1\le j\le k\right\}$ depend on the noncentrality parameters $\left\{{\lambda }_j={\kappa }_j;1\le j\le k\right\}$ , respectively.
2. In particular, the alternatives most likely and least likely to be discerned in terms of power are ${\theta }_1$ and ${\theta }_k$ having powers $\Psi \left(k_1\right)$ and $\Psi \left(k_k\right)$ , respectively.
3. Consider alternatives $\{{\gamma }_j\in S_p\left(Q_2\right)\}$ standardized to unit lengths. Then bounds on powers at these local alternatives are given by
   $\Psi \left({\kappa }_{r+s}\right)\le \{\Psi \left({\lambda }_j\right)forevery{\gamma }_j\in S_p\left(Q_{{}^2}\right)\}\le \Psi \left({\kappa }_r^{}\right)$
4. Suppose that $k_r$ is repeated s times as in the spectral resolution (2) for $\Omega$ . Then for each alternative $\{{\gamma }_j=Q_2{a}_j\in S_p\left(Q_2\right),with{a^{\prime }}_j=\left[a_j{}_{{}_1},\dots ,a_j{}_{{}_s}\right]\}$ , the noncentrality parameter is $\left\{{\lambda }_j={\kappa }_r{a^{\prime }}_j{a}_j\right\}$ , with corresponding power $\left\{\Psi \left({\lambda }_j\right):{\gamma }_j\in S_p\left(Q_{{}_2}\right)\right\}$ .

Proof: Conclusion (i) follows directly from $\lambda \left({\theta }_j\right)={{\theta }^{\prime }}_j{\Xi }^{-1}{\theta }_j={{\theta }^{\prime }}_j\left({\displaystyle {\sum }_{i=1}^k{\kappa }_i{q}_i{q^{\prime }}_i}\right){\theta }_j={\kappa }_j$ since ${\theta }_j=q_j$ , whereas $\left\{{q^{\prime }}_j{q}_i=0;i\ne j\right\}$ and ${q^{\prime }}_j{q}_j=1$ by orthonormality. Conclusion (ii) follows directly from variational properties of Rayleigh quotients as in Lemma A.1(i) of the Appendix. In like manner conclusion (iii) follows from Lemma A.1(ii) as variational properties over subspaces. Conclusion (iv) follows from (iii) since $\left\{{q^{\prime }}_j{q}_i=0;i\ne j\right\}$

Remark 1. The directed alternatives ${\delta }_1^{\text{'}}=\left[1,1,0,\mathrm{...},0\right]$ and ${\delta }_2^{\text{'}}=\left[1,1,\mathrm{...},1\right]$ were featured earlier as discrepancies in the first two coordinates of $\left(\delta -{\delta }_0\right)$ , and as deviations about the equiangular line in ${ℝ}^k$ . Let ${\Xi }^{-1}=\left[{\xi }^{ij}\right]$ . Then powers $\Psi \left(\lambda \right)$ at these alternatives will depend on ${\lambda }_1={\displaystyle {\sum }_{i=1}^2{\displaystyle {\sum }_{j=1}^2{\xi }^{{}_{ij}}}}$ at ${{\delta }^{\prime }}_1=\left[1,1,0,\mathrm{...},0\right]$ , and on ${\lambda }_2={\displaystyle {\sum }_{i=1}^k{\displaystyle {\sum }_{j=1}^k{\xi }^{ij}}}$ at ${{\delta }^{\prime }}_2=\left[1,1,\mathrm{...},1\right]$ .

Corollary 1. On specializing the location–scale parameters $\left(\delta ,\Xi \right)$ , Theorem 1 applies verbatim as follows.

(i) Hotelling [2] $T^2:\left(\delta ,\Xi \right)=\left(\mu ,\Sigma \right);L\left(T^2\right)=T_k^2(\cdot ;n-1,\lambda )$ , the power $\Psi \left(\lambda \right)$ depending on $\lambda =n\left(\mu -{\mu }_0\right)\text{'}{\Sigma }^{-1}\left(\mu -{\mu }_0\right)$ .

(ii) General Linear Models: $\left(\delta ,\Xi \right)=\left(\beta ,{\left(X\text{'}X\right)}^{-1}\right);L\left(F\right)=F(\cdot ;k,n-k,\lambda )$ , the power $\Psi \left(\lambda \right)$ depending on $\lambda =\left(\beta -{\beta }_o\right)\text{'}X\text{'}X\left(\beta -{\beta }_o\right)/{\sigma }^2$ ,

Proof: The noncentral distribution $F(\cdot ;k,n-k,\lambda )$ clearly satisfies the assumptions of Theorem 1 on identifying $\left(\delta ,\Xi \right)=\left(\beta ,{\left(X\text{'}X\right)}^{-1}\right)$ as claimed. Similarly in testing $H_0:\mu ={\mu }_0$ against ${{\rm H}}_1:\mu \ne {\mu }_0$ , Hotelling’s $T^2$ inherits these properties through the conversion $L\left(\frac{\left(\nu -k+1\right)T^2}{k\nu }\right)=F(\cdot ;k,\nu -k+1,\lambda )$ . With $\lambda =n\left(\mu -{\mu }_0\right)\text{'}{\Sigma }^{-1}\left(\mu -{\mu }_0\right).$

Remark 2. Note that alternatives $\left\{{\theta }_j=q_j;1\le j\le k\right\}$ of unit lengths give noncentrality parameters $\left\{{\kappa }_j;1\le j\le k\right\}$ . If instead the directed alternatives are $\left\{{\theta }_j=c_j{q}_i;1\le j\le k\right\}$ , then the noncentrality parameters will be $\left\{c_j^2{\kappa }_j;1\le j\le k\right\}$ .

Remark 3 Note that the foregoing developments are for the general case that $\Omega =Q{D}_{\kappa }Q\text{'}$ is anisotropic with $\left\{{\kappa }_1\ge \dots \ge {\kappa }_k>0\right\}$ . If isotropic, then the following applies.

Definition 1. The model $\Xi$ is called isotropic if and only if $\Xi =d{I}_k$ , in which case power functions are directionally invariant, not depending on directions of alternatives in ${ℝ}^k$ . Add: Bounds on ARLs from restricted variation.

Sphericity

The density for ${{\rm N}}_k\left(\mu ,\Sigma \right)$ has spherical contours for the case that $\Sigma =d{I}_k$ , i.e., the model is isotropic. Sample evidence regarding the isotropy of $T^2$ is available. Mauchly [3] derived the Likelihood Ratio test for sphericity, namely, ${{\rm H}}_0:\Sigma =d{I}_k$ against ${{\rm H}}_1:\Sigma \ne d{I}_k$ . A contemporary test utilizes the modified statistic

$L{R}_M=-\left(\nu -\frac{2k^2+k+2}{6k}\right)ln\left[\frac{k^k\left|S\right|}{{\left(trS\right)}^k}\right]$ (4)

taking $S$ as the sample dispersion matrix from n observations, rejecting at level α for $L{R}_M>c_{\alpha }$ with $v=n-1$ and with $c_{\alpha }$ as the upper percentile of the central distribution ${\chi }^2\left(k\left(k+1\right)/2-1,0\right)$ . See, for example, Rencher [4].

Design Reversals

Developments thus far are predicated in part on the desirability to identify alternatives having varying powers of discernment. These include the most likely and least likely as in Theorem 1(ii). If the least likely is deemed to be of greatest interest, it remains to ask whether it might serve instead as the most likely alternative. In the context of designed experiments the answer is affirmative, as the intrinsic structure offers a venue for modifying a given design so as to achieve these ends. Details follow.

Consider the model $\left\{Y=\left[1_n, X\right]\left[\alpha ,\beta \text{'}\right]\text{'}+\epsilon \right\}$ with $X$ centered such that ${1^{\prime }}_{n}X=0$ , where location–scale parameters for $\widehat{\beta }$ are $\left(\beta ,\Xi \right)$ with $\Xi ={\left(X\text{'}X\right)}^{-1}$ as in Corollary 1(ii). In particular, the test for ${{\rm H}}_0:\beta ={\beta }_o$ against ${{\rm H}}_1:\beta \ne {\beta }_o$ utilizes $F=\left(\widehat{\beta }-{\beta }_o\right)\text{'}X\text{'}X\left(\widehat{\beta }-{\beta }_o\right)/k{S}^2$ with $S^2$ as the residual mean square and with noncentrality $\lambda =\left(\beta -{\beta }_o\right)\text{'}X\text{'}X\left(\beta -{\beta }_o\right)/{\sigma }^2$ , where it often suffices to take ${\sigma }^2=1.0$ . For $X\in {\mathbb{F}}_{n\times k}$ its singular decomposition, followed by $X\text{'}X$ , is

$X=P{D}_{\delta }Q\text{'}={\displaystyle \sum _{i=1}^k{\delta }_i{p}_i{q^{\prime }}_i},X\text{'}X=Q{D}_{\kappa }Q\text{'}$ (5)

with $P=\left[p_1,\dots ,p_k\right]$ as its left–singular vectors, $D_{\delta }$ as its singular values, and columns of $Q\in {\mathcal{O}}_k$ as its right–singular vectors. Clearly $\left\{{\kappa }_j={\delta }_i^2;1\le i\le \kappa \right\}$ . Our principal reconstruction is articulated in the following.

Theorem 2. Let $\pi \left(\delta \right)$ be a permutation operator reversing the ordered array $\left[{\delta }_1\ge \dots \ge {\delta }_k\right]$ to $\left\{{\delta }_k\le \left\{{\delta }_2,\dots ,{\delta }_{k-2}\right\}\le {\delta }_1\right\}$ , and let $D_{\pi }=Diag\left({\delta }_k,{\delta }_2,\dots ,{\delta }_{k-2},{\delta }_1\right)$ . Next construct

$X_{\pi }=P{D}_{\pi }Q\text{'}={\delta }_k{p}_1{q^{\prime }}_1+{\displaystyle {\sum }_{i=2}^{k-1}{\delta }_i{p}_i{q^{\prime }}_i}+{\delta }_1{p}_k{q^{\prime }}_k$

such that pairs $\left({\delta }_k,q_1\right)$ and $\left({\delta }_1,q_k\right)$ are realigned.

Conclusion: The most likely and least likely alternatives for design $X_{\pi }$ are reversed from those of $X$ , so that ${\theta }_1=q_k$ now is most likely with power $\Psi \left({\kappa }_1\right)$ depending on ${\kappa }_1$ , and ${\theta }_k=q_1$ least likely with power $\Psi \left({\kappa }_k\right)$ depending on ${\kappa }_k$ .

Proof: Clearly the conventional reordering of eigenvalues gives

${X^{\prime }}_{\pi }{X}_{\pi }={\kappa }_1{q}_k{q^{\prime }}_k+{\displaystyle \sum _{i=2}^{k-1}{\kappa }_i{q}_i{q^{\prime }}_i}\text{+}{\kappa }_k{q}_1{q^{\prime }}_1$ (6)

and the conclusion follows on applying Theorem 1(ii) in the context of Corollary 1(ii).

Remark 4. Variations on $X_{\pi }$ are apparent. Any permutation of $\left\{{\delta }_2,\dots ,{\delta }_{\kappa -1}\right\}$ gives the same conclusion. In addition, any pairs $\left\{\left({\delta }_i,q_i\right),\left({\delta }_j,q_j\right)\right\}$  may be selected in like manner as most likely and least likely to be discerned. Note, however, that these tools are available in the case of first–order designs.

Studies in Hotelling [2] $T^2$ and second–order response models are given, to illustrate Theorem 1 and Corollary 1. Moreover, an example design serves to illustrates the Theorem 2 reversal of most likely and least likely alternatives.

Hotelling’s $T^2$ Tests

We reexamine the role of calcium in the growth of turnip greens, using data as reported in Kramer et al. [5]. In each of 29 experimental plots the plant calcium ( $Y_1$ ) was determined, and the soil calcium was assayed as available ( $Y_2$ ) and exchangeable ( $Y_3$ ) calcium. The units all are milliequivalents per hundred grams. Horticultural specialists expect these to run at about 15.00, 6.00 and 2.85 units, respectively. The sample means are $\overline{Y} \text{'}=\left[17.97, 4.39, 2.46\right];$ and the sample dispersion matrix is $S$ with inverse $S^{(-1)} = Q{D}_{k}Q\text{'}$ in spectral form, as listed in

$Q=\left[\begin{array}{l}0.010240.22479-0.97435\\ 0.05973-0.97280 -0.22381\\ -0.99816 -0.05591 -0.02339\end{array}\right]$

where $D_k = Diag (5.61102, 0.04857, 0.00416)$ The data are ill–conditioned, with condition number

$c_1\left(S\right)=1,348.80$ :

The statistic reported is $T^2 = n (\overline{Y}- {\mu }_0) \text{'} S^{-1}(\overline{Y} -{\mu }_0) = 24.97$ with ${\mu }_0{}^{\prime }= 15.00, 6.00, 2.85$ , rejecting at level $\alpha = 0.01$ the hypothesis $H_0: \mu ={\mu }_0$ in favor of some $H_1: \mu \ne {\mu }_0$ . Indeed, the $p$ –value is $P(T^2> 24.97 | H_0) = 0.000751$ with $C_{\alpha } = [14.980]$ . On applying Corollary 1(i) with $S$ in lieu of $\Sigma$ , the columns of $Q= \left[q_{{}_1}, q_{{}_2,} q_{{}_3}\right]$ are taken as successive alternatives to $\left(\mu - {\mu }_{{}_0}\right)$ , namely

$\left[\begin{array}{l}{\mu }_{{}_1- 15.00}\\ {\mu }_2- 6.00\\ {\mu }_{3 - 2.85}\end{array}\right] \in \left[\begin{array}{l}0.010240.22479-0.97435\\ 0.05973-0.97280 -0.22381\\ -0.99816 -0.05591 -0.02339\end{array}\right]$

where the dominant terms are in bold type. The noncentrality parameters $\{{\lambda }_i =n{\kappa }_i; 1\le i \le 3\}$ are {162.720, 1.409, 0.121}, and taking $\alpha =0.05$ and $C_{\alpha }=9.612,$ powers at these alternatives are

$P((T^2|{\lambda }_1)>9.612= 1.0000,P((T^2|{\lambda }_2)>9.612=0.1316, P((T^2|{\lambda }_3)>9.612) = \mathrm{0.0562.}$

Accordingly, $T^2$ has essentially unit power to distinguish the hypothetical deviation $({\mu }_3- 2.85)$ from -0.99816, since the discrepancies 0.01024 for $({\mu }_1- 15.00)$ and 0.05973 for $({\mu }_2- 6.00)$ in $q_1$ are negligible. Similarly, $T_{{}_2}$ is marginally able to distinguish [( ${\mu }_1$ −15.00), ( ${\mu }_2$ −6.00)] from [0.22479, −0.97280] with power 0.1316, but is virtually unable to separate [( ${\mu }_1$ −15.00), ( ${\mu }_2$ −6.00)] from [−0.97435, −0.22381] with negligible power of 0.0562. In short, the latter suggests [14.03, 5.78] to be plausible values for $[{\mu }_1,{\mu }_2]$ .

This is an example, as seen subsequently also, where elements of $Q= \left[q_{{}_1}, q_{{}_2,} q_{{}_3}\right]$ , especially $q_{{}_1}$ , convey useful information in regard to the objectives of the study. In summary, details regarding directed alternatives, enabled here by Theorem 1 and Corollary 1(i), go beyond conventional useage for $T^2$ .

Hotelling’s $T^2$ Charts.

Multivariate diagnostics figure prominently in Statistical Process Control (SPC), as reviewed subsequently. In monitoring the manufacture of bomb sights during World War II, Hotelling [6] devised $T^2$ charts for multivariate means in $\left\{R\right\}\text{^}\left\{K\right\}$ . Here successive values $\{T_i^2; i = 1, 2,\mathrm{...}\}$ are charted against time, where the chart signals the process to be out–of–control at level $\alpha$  whenever $T_i^2> C_{\alpha }.$ Moreover, with power $\psi (\lambda ) = P ((T_i^2|\lambda ) > C_{\alpha }),$ the Average Run Length (ARL) of time–to–signal is $ARL = 1/\psi (\lambda )$ . To monitor the mean $\mu$ against its target value ${\mu }_0$ , successive samples of size n yield $\left(\overline{Y_i}, S_{i }\right)$ , together with $T_i^2=n(\overline{Y_i}-{\mu }_0{)}^{\prime } S_i^{-1}(\overline{Y_i}-{\mu }_0)$ having the $T_k^2(\therefore n-1,\lambda )$ distribution with $\lambda = n \left(\mu -{\mu }_0\right)\text{'}{\Sigma }^{-1}\left(\mu -{\mu }_0\right)$ . Phase I in SPC is set to establish base line process capabilities, to include parameter estimation, followed in Phase II by implementing the control charts themselves.

To continue, consider the data of Quesenberry [7] to be in Phase I, comprising n=30 records of 11 quality characteristics indexed in time–order of production. Following Williams et al. [8], dimensions are reduced on selecting the first k=5 quality characteristics, namely, $[Y_1,\mathrm{...},Y_5]$ having means $[{\mu }_1,\mathrm{...},{\mu }_5]$ , respectively. As in Section Hotelling’s $T^2$ Tests we take $S$ in lieu of $\sum$ , finding the spectral resolution $S^{-1} = Q{D}_{\kappa }Q\text{'}$ as reported in Table 1. The data are seen to be highly ill–conditioned, with condition number $c_1\left(S\right)=3,945.64$ . In keeping with Corollary 1(i), five directed alternatives comprise the columns of $Q = [q_1,\mathrm{...},q_5]$ in Table 1, where dominant elements again are in bold type. In particular, this example shows $\left\{q_1,\mathrm{...},q_5\right\}$ to be separately informative per se, as each corresponds essentially to deviations in $\left({\mu }_i-{\mu }_{i0}\right)$ for observations $\left\{Y_1,Y_3,Y_5,Y_2,Y_4\right\}$ , respectively, since values other than those in bold type are negligible.

Taking $\{{\lambda }_i = n {\kappa }_i; 1 \le i \le 5\}$ with $n=30,\alpha =0.05$ , and critical value $C_{\alpha } = 15.097$ , powers against these directed alternatives are listed in Table 1. These show that deviations in the directions of $\{{\mu }_1,{\mu }_2,{\mu }_3,{\mu }_5\}$ essentially would be discerned with power at least 0.9882, whereas power in the direction of ${\mu }_4$ would be diminished to 0.2367. Note, however, that these values are inflated by the value n=30 in Phase I. Samples much smaller in size ordinarily would be taken in Phase II, say n=8 in this example. Then the powers corresponding to $\{{\lambda }_i= 8 k_i; 1 \le i \le 5\}$  at α=0.05 are given subsequently in Table 1. In short, with $ARL=1/\psi \left(\lambda \right)$ as in the final row of Table 1 with samples of size n=8, these charts would detect changes in ${\mu }_1$ and ${\mu }_3$ immediately on average, but less responsive otherwise with $ARLsof\left(5.41, 5.62, 15.50\right)for\left({\mu }_5, {\mu }_2, {\mu }_4\right)$ , respectively.

Remark 5. This example goes beyond conventional uses of $T^2$ charts, in demonstrating that the capacity of a given chart to detect alternatives may differ widely across alternatives in ${ℝ}^5$ of compelling practical interest. In short, each of five ARLs pertains here to an informative one–dimensional alternative.

Second–Order Designs

Second–order models of type

$\{Y_i= {\beta }_0 + {\beta }_1 X_{1i}+ {\beta }_2{X}_{2i}+{\beta }_{11}{X}_{1i}^2+{\beta }_{22}{X}_{2i}^2+{\beta }_{12}{X}_{1i}{X}_{2i}+ {\epsilon }_{i;} 1 \le i\le n\}$ (8)

are considered having zero mean, uncorrelated errors with variance ${\sigma }^2$ . In a typical setting the yield ( $Y_i$ ) of a chemical process is examined at specified reaction time $(X_{1i})$ and temperature $(X_{2i})$ . Small designs of historical consequence are the Central Composite (CCD) designs of Box et al. [9], having design points as listed in Table 2.

Proceeding as in Corollary 1(ii), we seek spectral values for the Fisher Information Matrix $X^{\prime }X$ , specifically, the eigenvalues and eigenvectors as listed in Table 3, with dominant terms again in bold type. Powers at these directed alternatives follow on taking ${\mu }_3$ as surrogates in the noncentrality parameters for $F = (\stackrel{⌢}{\beta }-{\beta }_o{)}^{\prime }{X}^{\prime }X (\stackrel{⌢}{\beta } - {\beta }_o)/ k{S}^2$ with $k=6$ and $S^2$ as the residual mean square. Owing to only two degrees of freedom for error, computations replaced $S_i^2 by {\sigma }^2 = 1.0,$ then powers were determined from scaled noncentral $X^2(6,{\kappa }_i)$ distributions with noncentrality parameters as listed in Table 3.

Arranged in decreasing order of their powers, these are $q_1$ with power 0.9765; $\{q_2,q_3,q_4\}$ each with power 0.5307; and $q_5$ and $q_6$ as alternatives with powers 0.2698 and 0.0775. Here elements of $Q$ are separately informative: $q_2$ for discrepancies between $({\beta }_{11},{\beta }_{22})$ and their hypothetical values; and $(q_3,q_4,q_5)$ for discrepancies between $\left({\beta }_1,{\beta }_2,{\beta }_{12}\right)$ and their hypothetical values, respectively. To continue, observe that the eigenvalue ${\kappa }_2=8.0000$ is repeated three times. On applying Theorem 1(iv) in the context of Corollary 1(ii), we see that all standardized elements in $S_p\left(q_2,q_3,q_4\right)$ have power 0.5307. These include, in addition to $q_3$ for ${\beta }_1$ and $q_4$ for ${\beta }_2$ , the standardized sums

${\theta }_1 = (q_3+q_4)/\sqrt{2 }= [0, 1, -1, 0,0,0{]}^{\prime }/\sqrt{2}$ (9)

${\theta }_2 = (q_2+ q_3 + q_4)/\sqrt{3 }= [0, 1, -1, 0,7071, -0.7071, 0{]}^{\prime }/\sqrt{3}$ (10)

for example, with ${\theta }_1$ for the discrepancy between $[ ({\beta }_1- {\beta }_{10}), ({\beta }_2- {\beta }_{20})]$ and $[ 1, -1] /\sqrt{2}$ .

In summary, the directed second–order alternatives treated here are innovations not found in classical linear inference. Instead, these are enabled by Theorem 1 and Corollary 1(ii). Again the elements of $Q$ , especially $\{q_2,\mathrm{....}, q_5\},$ are separately informative about coefficients of the model (4.2). Their simple and revealing structure may be attributed to the symmetry and balance of CCD designs.

Design Reversals

Begin with $\{ Y = [1_{n,} X][{\beta }_{0 },{\beta }^{\prime }{]}^{\prime }+\epsilon \}$ with $X_0=\left[1_n,X\right]$ and $X$ in centered form as in Section 3.2, with ${\beta }^{\prime } = [{\beta }_1,{\beta }_2,{\beta }_3],$ having the design $X = PDiag ({\delta }_1,{\delta }_2,{\delta }_3) Q^{\prime }$ as listed in Table 4. Construct $X_1 = PDiag ({\delta }_3,{\delta }_2,{\delta }_1)Q^{\prime }$ on permuting singular values, but retaining the left and right singular vectors.

To continue, take the right–singular vectors, now $Q = [{\theta }_1, {\theta }_2, {\theta }_3],$ as directed alternatives to $(\beta - {\beta }_o{)}^{\prime } = [({\beta }_1-{\beta }_{10}),$ $({\beta }_2 - {\beta }_{20}), ({\beta }_3-{\beta }_{30})],$ together with $k^{\prime } = [14.5909, 4. 4066, 0.2828]$ as $\{ {\kappa }_i = {\delta }_i^2; 1 \le i \le 3\}$ . Here the level 0.05 critical value is 6.5914; then powers are determined in turn from $F (\therefore 3, 4, {\kappa }_i)$ together with the power functions $\{ \psi (\lambda ) = P(F > 6.5914| {\kappa }_i); 1 \le i \le 3\}$ having values listed in the final row of Table 4.

In particular, discovering alternatives $(\beta -{\beta }_o{)}^{\prime }$ in the direction ${\theta }_3 = -[0.6921, 0.1865, 0.6973]$ is seen to be unlikely, with power 0.0577. On the other hand, suppose instead that it is critical in context to discover alternatives in the negative orthant of ${ℝ}^3$ . Then the design $X_1$ serves to reverse these so that alternatives in the directions of $\{ {\theta }_3, {\theta }_2, {\theta }_1\}$ are now detected with probabilities $\left[0.4947, 0.1828, 0.0577\right]$ , respectively.

Further properties of the design $X$ and its reversal $X_1$ deserve mention. Observe that ${X^{\prime }}_0{X}_0 =Diag (n, X^{\prime }X),$ whereas $X^{\prime }X = Q Diag({\kappa }_1,{\kappa }_2,{\kappa }_3)Q^{\prime }$ and ${X^{\prime }}_1 X_1 = Q_R Diag({\kappa }_1,{\kappa }_2,{\kappa }_3){Q^{\prime }}_R$ , where now $Q_R = [q_3,q_2,q_1]$ following the convention that $\{{\kappa }_1 \ge {\kappa }_{2 }\ge {\kappa }_3\}$ remain ordered. Clearly $V(\stackrel{⌢}{\beta }|X) = {\sigma }^2{(X^{\prime }X)}^{-1}$ and $V(\stackrel{⌢}{\beta }|X_1) = {\sigma }^2{({X^{\prime }}_1{X}_1)}^{-1}$ differ, where their diagonal elements are listed as variances in Table 5. However, their eigenvalues are identical by construction, as are their $\{A,D,E\}$ efficiency indices as the trace, determinant, and largest eigenvalues of ${({X^{\prime }}_0{X}_0)}^{-1}$ under both designs $X and X_1$ .

This study reexamines the concept of directional invariance, or isotropy, for distributions on ${ℝ}^k$ having location–scale parameters $(\delta , \Xi )$ . Powers of tests for $H_0: \delta = {\delta }_0$ against $H_0: \delta \ne {\delta }_0$ often depend on noncentrality parameters of type $\lambda = (\delta - {\delta }_0{)}^{\prime }{\Xi }^{-1}(\delta - {\delta }_0)$ . The spectral decomposition of $\Xi$ supports the identification of directed alternatives in directions determined by the eigenvectors of $\Xi$ , to encompass the alternatives most likely and least likely in a given study. Powers of these types are independent of direction if and only if $\Xi = {\sigma }^2{{\rm I}}_k$ . Applications are drawn in the use of Hotelling [2] $T^2$ in multivariate samples, and of F-tests in linear models. Case studies are given where this approach leads to the discovery of further insight regarding the natural parameters of a problem.

One concept of directional invariance figures prominently in the SPC literature. The following is excerpted from Linna et al. [10].

“It is well known that the ARL performance of multivariate SPC procedures depends heavily on the covariance structure of the observed data. See, for example, Mason et al. [11]. Further, it has been noted by Pignatiello et al. [12] that many multivariate procedures, including the ${\chi }^2$ chart, Hotellings $T^2$ chart, and most of the multivariate CUSUM charts, are directionally invariant. The performance of the multivariate EWMA chart proposed in Lowry et al. [13] is also directionally invariant. Lowry et al. [14] and others also note the directional invariance of many of these multivariate control charting methods. Directional invariance means that the performance of a procedure does not depend on the specific direction in p-space of a shift in the mean vector of the process variables being monitored. Instead, performance of a directionally invariant procedure depends only on the statistical (or Mahalanobis) distance between the in-control mean vector ${\mu }_0$ and the out-of-control mean vector under consideration, ${\mu }_1$ .” (Italics supplied.) That is, $D_{\Xi }^2({\mu }_0, {\mu }_1) = ({\mu }_0- {\mu }_1{)}^{\prime }{\Xi }^{-1}({\mu }_0-{\mu }_1)$ .

Unfortunately, this notion of directional invariance is grossly misleading, is antithetical to the very concept of invariance as in our Definition 1, at best is a misnomer, and in any event deserves to be clarified in the SPC literature. In fact, such essentials as power functions and ARLs do indeed depend on directions of alternatives, as seen in Section 4.2 as counter examples, unless the model is isotropic.

On the other hand, a disclaimer of Tsui et al. [15] should be noted: “There is no reason in practice, however, for a shift to $\mu = \mu *$ to be always considered as important as a shift to $\mu = \mu **$ just because the corresponding values of the noncentrality parameters are equal.” Our study represents a substantial elaboration on this point.

Rayleigh Quotients.

At issue are variational properties of quadratic forms of type $Q(\upsilon ) = {\upsilon }^{\prime } \Omega \upsilon /{\upsilon }^{\prime } \upsilon ,$ known as Rayleigh Quotients; see Bellman [16]. Write $\Omega = {\Sigma }_{i=1}^k {\kappa }_i{q}_i{q^{\prime }}_i = Q{D}_k{Q}^{\prime }$ in its spectral form with $Q \in {\mathcal{O}}_k$ so that $\{{q^{\prime }}_i\Omega q_{i }= {\kappa }_i; 1 \le i \le k\}$ . Further partition $Q = [Q_1,Q_2,Q_3]$ of orders $\{ (k\times (r-1)), (k\times s) (k \times d)\}$ with $d = k - r-s + 1,$ and partition $D_{\kappa } = Diag(D_{{\kappa }_1},D_{{\kappa }_2},D_{{\kappa }_3})$ conformably. Then

$\Omega = Q_1{D}_{{\kappa }_1}{Q^{\prime }}_1 + Q_2{D}_{{\kappa }_2}{Q^{\prime }}_2 + Q_3{D}_{{\kappa }_3}{Q^{\prime }}_3$ (11)

where, in particular, $Q_2 = [q_r,\mathrm{...},q_{r+s}] and D_{k_2} = Diag ({\kappa }_r,\mathrm{...},{\kappa }_{r+s})$ . Essentials follow.

Lemma A.1 Consider the positive definite form ${\upsilon }^{\prime }\Omega \upsilon /{\upsilon }^{\prime } \upsilon with \Omega$ as in expression (11).

(i) Variational properties of $Q(\upsilon )$ as u varies over ${ℝ}^k$ are

${\kappa }_k\le \frac{{\upsilon }^{\prime }\Omega \upsilon }{{\upsilon }^{\prime }\upsilon } \le {\kappa }_1$ (12)

where the lower and upper limits are attained at $\upsilon = q_k$ and $\upsilon = q_1$ , respectively.

(ii) Variational properties of $Q(\upsilon )$ as u varies over $S_p(Q_2)$ are

${\kappa }_{r+s }\le \frac{{\upsilon }^{\prime }\Omega \upsilon }{{\upsilon }^{\prime }\upsilon } \le {\kappa }_r$ (13)

where the lower and upper limits are attained at $\upsilon = q_{r+s }$ and $\upsilon = q_r$ , respectively.

Proof: Conclusion (i) is given in Bellman [16], where the limits are attained as given since ${\kappa }_k= {q^{\prime }}_{k }\Omega q_k$ and ${\kappa }_1 = {q^{\prime }}_1\Omega q_1$ . To see conclusion (ii), $\upsilon \in S_p (Q_2)$ may be represented as $\upsilon = Q_2 a with a^{\prime } = [a_r,\mathrm{...}, a_{r+s}]$ . Then for $\upsilon \in S_p (Q_2)$ , (A.1) gives ${\upsilon }^{\prime }\Omega \upsilon = a^{\prime }{Q^{\prime }}_2{Q}_2{D}_{{\kappa }_2}{Q^{\prime }}_2{Q}_{2}a = {\Sigma }_{i=r}^{r+s} {\kappa }_i{a}_i^2$ . The lower and upper limits for $\upsilon \in S_p (Q_2)$ follow as in conclusion (i) for $\upsilon \in {ℝ}^k$ , except that now these are attained as given since ${q^{\prime }}_{r+s}\Omega q_{r+s} = {\kappa }_{r+s}$ and ${q^{\prime }}_r \Omega q_r= {\kappa }_r$ .