## 2.1  Special  Case: Known Mean

## Source details

**Canonical URL:** [2.1  Special  Case: Known Mean](https://www.imf.org/-/media/files/publications/wp/2023/english/wpiea2023257-print-pdf.pdf)

## Other formats

- [Markdown version](/-/media/files/publications/wp/2023/english/wpiea2023257-print-pdf.pdf.md)
- [Structured JSON version](/-/media/files/publications/wp/2023/english/wpiea2023257-print-pdf.pdf.json)

---

### Introduction and motivation
- Problem: Estimating a covariance matrix Σ: p×p and its inverse in high-dimensional settings p > n is central to financial portfolio selection, macroeconomic forecasting, Generalized Method of Moments, and Principal Component Analysis.
- Limitations of existing shrinkage estimators (Ledoit and Wolf (2004); Chen et al. (2009)): they target average variance and can perform poorly when diagonal elements of Σ exhibit substantial variation (example: GDP versus small-industry outputs differing by a hundredfold).
- Proposal: a shrinkage estimator that targets the diagonal elements of the sample covariance matrix (diag(S)), extending the Oracle Approximating Shrinkage estimator (OAS) to better accommodate large variation in variances.

### Key methodological points
- Sample covariance matrix (known mean case):
  - S := 1/n ∑_{i=1}^n x_i x_i^T, with S_mm > 0 for all m = 1,...,p and Σ > 0 (known-mean specification replaces 1/(n−1) by 1/n).
- Linear shrinkage form:
  - Ŝ(ρ) := (1−ρ) S + ρ T, where T is a target matrix; here T = diag(S).
  - If T positive definite and ρ ∈ (0,1], then Ŝ(ρ) is positive definite even when S is degenerate.
- Oracle estimator (known Σ) minimizes MSE:
  - ρ_OD(Σ,T) := arg min_{ρ∈R} E[∥Ŝ(ρ) − Σ∥^2], with ∥A∥^2 := tr(A^T A) = ∑_{i,j} A_{i,j}^2.
  - Theorem 1 (closed form): ρ_OD(Σ,T) = E[tr(Σ−S)(T−S)] / E[∥T−S∥].
- For joint normal x_i ∼ N(μ,Σ) and T = diag(S), Oracle shrinkage (known-mean specification adjustments noted in Theorem 3):
  - ρ_OD(Σ) = 1 / [1 + n φ(Σ)] ∈ (0,1],
  - where φ(Σ) := [tr(Σ^2) − tr(diag(Σ)^2)] / [tr(Σ^2) + tr(Σ)^2 − 2 tr(diag(Σ)^2)] ∈ [0,1).

### OASD iterative approximation (known-mean adjustments)
- Sample analogue replacing Σ by S in φ gives ρ_OD := 1 / [1 + n φ].
- Oracle-Approximating Shrinkage with Diagonal target (OASD) built from iteration:
  - Σ_j = (1−ρ_j) S + ρ_j diag(S)
  - ρ_{j+1} = [tr(Σ_j S) − 2 tr(diag(Σ_j)^2) + tr(Σ_j)^2] / [n tr(Σ_j S) − (n+1) tr(diag(Σ_j)^2) + tr(Σ_j)^2]
- Theorem 2 (convergence): For any ρ_0 ∈ (0,1), sequence {ρ_j} converges monotonically to
  - ρ_OASD := min{ 1/[(n+1) φ], 1 } ∈ (0,1].
- Property: ρ_OASD ∈ (0,1] ensures S_OASD := (1−ρ_OASD) S + ρ_OASD diag(S) is positive definite; the min operator arises from convergence rather than manual imposition.

### Simulation setup (Section 3.1) — known-mean experiments
- Dimension: p = 100.
- Sample size n varies from 6 to 30.
- True covariance Σ constructed as Σ = Λ Γ Λ, where:
  - Γ is a correlation matrix with decaying off-diagonals Γ_{ij} = γ^{|i−j|}, γ ∈ {0,...,.9},
  - Λ diagonal with half variables scaled by 10 and half by 1, with parameter sd controlling variation across diagonal elements:
    - Λ = diag([10 ... 1 sd ... 0 sd ...]) with sd varying from 1 to 20 to generate large variation in scales.
- Draws: x_i ∼ N(0,Σ), repeated B = 5000 times.
- Performance metric: Percentage Relative Improvement in Average Loss (PRIAL):
  - PRIAL(Ŝ) := (1 − [∑_{b=1}^B ∥Ŝ^{(b)} − Σ∥^2] / [∑_{b=1}^B ∥S^{(b)} − Σ∥^2]) × 100.
  - PRIAL = 100 when Ŝ coincides with Σ; PRIAL = 0 when Ŝ as poor as S; negative when worse than S.
- Methods compared (known mean setting):
  - LW (Ledoit and Wolf (2004)):
    - S_LW := (1−ρ_LW) S + ρ_LW tr(S)/p I,
    - ρ_LW := min{ [∑_{i=1}^n ∥x_i x_i^T − S∥^2] / [n^2 (tr(S^2) − tr(S)^2 / p)], 1 }.
  - OAS (Chen et al. (2009)):
    - S_OAS := (1−ρ_OAS) S + ρ_OAS tr(S)/p I,
    - ρ_OAS := min{ [1 − 2/p] [tr(S^2) + tr(S)^2] / [ (n+1 − 2/p)(tr(S^2) − tr(S)^2 / p) ], 1 } (modified formula; paper notes a typo in original).
  - SS (Schäfer and Strimmer (2005)):
    - S_SS := (1−ρ_SS) S + ρ_SS diag(S),
    - ρ_SS := min( [∑_{k≠l} dVar(r_{kl})] / [∑_{k≠l} r_{kl}^2], 1 ), where r_{kl} are sample correlations.
  - OD sample analogue: S_OD = Ŝ(ρ_OD(S)).
- Comparisons vary parameters {n, sd, γ}, holding medians fixed when varying one parameter at a time.

### Main simulation results (Section 3.2)
- Overall:
  - OASD (S_OASD) tends to produce higher PRIAL and a ρ that tracks the infeasible Oracle ρ_OD(Σ) more closely across variations in n, sd, and γ.
- Variation in scales (sd):
  - For most sd (except very small values), OASD exhibits higher PRIAL than average-variance target methods LW and OAS.
  - When sd is small, LW and OAS perform better (OASD is complementary rather than universally dominant).
  - OASD shows ~1 percent higher PRIAL than SS.
  - The sample-analogue OD estimator performs much poorer numerically than other methods in the simulation (difference between formulas with n vs n+1 can numerically change shrinkage substantially; e.g., n = 18 and φ ≈ .065 produces about half the value).
  - All methods show lower PRIAL as sd increases because targets with null off-diagonals approximate Σ worse; correspondingly shrinkage parameters decrease.
- Sparsity of correlation matrix (γ):
  - OASD exhibits higher PRIAL than other methods; improvement relative to LW and OAS can be up to 10 percent when Σ is sparser (γ small).
  - Intuition: as γ → 0 and Σ becomes diagonal, diag(S) target allows shrinking off-diagonals without distorting diagonal elements; average-variance targets face a trade-off.
  - OASD outperforms SS by up to 10 percent; OASD's ρ approximates ρ_OD(Σ) better than ρ_SS.
  - Differences shrink as Σ becomes denser (γ large).
- Sample size (n):
  - OASD performs best across sample sizes n.
  - On average, PRIAL(OASD) is 10 percent higher than LW and OAS and 2 percent higher than SS in the examined region.
  - Differences increase with n in the plotted region but ultimately vanish as n → ∞ as S → Σ.
  - ρ_OASD tracks ρ_OD(Σ) more closely for all n.

### Performance of inverse matrices (Section 3.3)
- Motivation: in many applications the inverse Σ^{−1} is the ultimate object of interest; shrinkage methods are designed to minimize MSE of Σ, not Σ^{−1}.
- Modified PRIAL for inverses:
  - PRIAL_INV(Ŝ) := (1 − [∑_{b=1}^B ∥{Ŝ^{(b)}}^{−1} − Σ^{−1}∥^2] / [∑_{b=1}^B ∥{S^{(b)}}^+ − Σ^{−1}∥^2]) × 100, where A^+ denotes generalized inverse.
- Findings:
  - The inverse of S_OASD tends to approximate Σ^{−1} better than other methods.
  - Intuition: when Σ is diagonal with disparate diagonal elements (e.g., 1 and 10), diag(S)^{-1} has diagonal entries nearer to Σ^{-1} than [tr(S)/p I]^{-1}.
  - PRIAL_INV for OD is not monotonic in γ and can be higher than both OASD and the Oracle in some regions; in other regions OD can be worse than the generalized inverse of the sample covariance.
  - OD and SS share the same diagonal target but differ in PRIAL_INV because OD’s shrinkage parameter can be much smaller than SS’s, especially when γ is small.

### Alternative method based on shrinking the correlation matrix (Section 3.4)
- Procedure:
  - Obtain sample correlation matrix R, apply existing shrinkage methods to R, and scale back the estimated correlation matrix by multiplying sample standard deviations.
  - For e ∈ {LWcorr, OAScorr, SScorr}, set S^e = diag(S)^{1/2} R^e diag(S)^{1/2}.
- Properties:
  - The diagonal elements of S^e equal the sample variances for all e ∈ {LWcorr, OAScorr, SScorr}, matching S_OASD.
  - ρ_SScorr = ρ_SS and S_SS = S_SScorr.
  - S^e = diag(S)^{1/2}{(1−ρ_e)R + ρ_e I}diag(S)^{1/2} = (1−ρ_e)S + ρ_e diag(S), so S^e lies in feasible set for diagonal-target shrinkage.
- Performance:
  - OASD still outperforms other methods in terms of MSE.
  - LWcorr and OAScorr exhibit lower MSE than their covariance-based counterparts but still lag behind OASD.
  - Reason: ρ_e may not be optimized to minimize MSE of the covariance matrix, while ρ_OASD approximates ρ_OD(Σ) that does minimize MSE.

### Summary conclusions and caveats
- Conclusions:
  - Targeting diagonal elements via OASD improves covariance estimation relative to average-variance targets when diagonal variation is substantial and when the true correlation matrix is sparse.
  - OASD yields better finite-sample approximations to the Oracle shrinkage weight ρ_OD(Σ) across variations in n, sd, and γ.
  - OASD also improves approximation to Σ^{−1} in many settings.
  - OASD is complementary to existing methods: when diagonal variation is small, average-variance-targeting methods (LW, OAS) may perform better.
- Caveats:
  - Results are based on a normality assumption. Normalization procedures, such as the Box-Cox transformation, may need to be used if the distribution of data deviate substantially from normality.
  - The methods leave the sample variances intact and only shrink the off-diagonal entries. Extensions can allow different levels of shrinkage for diagonal and off-diagonal entries; this is left for future research.

*Source: wpiea2023257-print-pdf — section 2.1 Special Case: Known Mean; sections 3, 3.1, 3.2, 3.3, and 3.4.*

### 2.1  Special  Case: Known Mean .........................................................................................

### 2.1  Special  Case: Known Mean

### Introduction and motivation
- Estimating a covariance matrix Σ:p×p and its inverse in high-dimensional settings p > n is central to applications such as financial portfolio selection, macroeconomic forecasting, Generalized Method of Moments, and Principal Component Analysis.
- Existing shrinkage estimators (Ledoit and Wolf (2004); Chen et al. (2009)) target average variance and can perform poorly when diagonal elements of Σ exhibit substantial variation (e.g., GDP versus small-industry outputs differing by a hundredfold).
- This paper proposes a shrinkage estimator that targets the diagonal elements of the sample covariance matrix (diag(S)), extending the Oracle Approximating Shrinkage estimator (OAS) to better accommodate large variation in variances.

### Key methodological points
- Sample covariance matrix (unknown mean case):
  - S := 1/(n−1) ∑_{i=1}^n (x_i − x̄)(x_i − x̄)^T, with S_mm > 0 for all m = 1,...,p and Σ > 0.
- Linear shrinkage form:
  - Ŝ(ρ) := (1−ρ) S + ρ T, where T is a target matrix; here T = diag(S).
  - If T positive definite and ρ ∈ (0,1], then Ŝ(ρ) is positive definite even when S is degenerate.
- Oracle estimator (known Σ) minimizes MSE:
  - ρ_OD(Σ,T) := arg min_{ρ∈R} E[∥Ŝ(ρ) − Σ∥^2], with ∥A∥^2 := tr(A^T A) = ∑_{i,j} A_{i,j}^2.
  - Theorem 1 (closed form): ρ_OD(Σ,T) = E[tr(Σ−S)(T−S)] / E[∥T−S∥].
- For joint normal x_i ∼ N(μ,Σ) and T = diag(S), Oracle shrinkage:
  - ρ_OD(Σ) = 1 / [1 + (n−1) φ(Σ)] ∈ (0,1], where
  - φ(Σ) := [tr(Σ^2) − tr(diag(Σ)^2)] / [tr(Σ^2) + tr(Σ)^2 − 2 tr(diag(Σ)^2)] ∈ [0,1).

### OASD iterative approximation
- Sample analogue replacing Σ by S in φ gives ρ_OD := 1 / [1 + (n−1) φ].
- Oracle-Approximating Shrinkage with Diagonal target (OASD) built from iteration:
  - Σ_j = (1−ρ_j) S + ρ_j diag(S)
  - ρ_{j+1} = [tr(Σ_j S) − 2 tr(diag(Σ_j)^2) + tr(Σ_j)^2] / [n tr(Σ_j S) − (n+1) tr(diag(Σ_j)^2) + tr(Σ_j)^2]
- Theorem 2 (convergence): For any ρ_0 ∈ (0,1), sequence {ρ_j} converges monotonically to
  - ρ_OASD := min{ 1/(n φ), 1 } ∈ (0,1].

### Properties of OASD
- ρ_OASD ∈ (0,1] ensures S_OASD := (1−ρ_OASD) S + ρ_OASD diag(S) is positive definite.
- The min operator in ρ_OASD arises from convergence rather than manual imposition.
- Unlike OAS, ρ_OASD formula does not contain the dimension p.

### Special case: known mean μ = 0 (Section 2.1)
- When μ = 0 and S := 1/n ∑_{i=1}^n x_i x_i^T, n in formulas (6), (8), and (11) is replaced by n+1.
- Theorem 3 (known mean results):
  - ρ_OD(Σ) = 1 / [1 + n φ(Σ)] ∈ (0,1],
  - ρ_OD := 1 / [1 + n φ] ∈ (0,1],
  - ρ_OASD := min{ 1/[(n+1) φ], 1 } ∈ (0,1].
- φ formula remains the same but with Σ and S defined under the known-mean specification.
- S_OASD := (1−ρ_OASD) S + ρ_OASD diag(S) is positive definite; the min operator again follows from convergence.

### Simulation setup (Section 3.1)
- Dimension: p = 100.
- Sample size n varies from 6 to 30.
- True covariance Σ constructed as Σ = Λ Γ Λ, where Γ is a correlation matrix with decaying off-diagonals Γ_{ij} = γ^{|i−j|}, γ ∈ {0,...,.9}, and Λ diagonal with half variables scaled by 10 and half by 1, with parameter sd controlling variation across diagonal elements:
  - Λ = diag([10 ... 1 sd ... 0 sd ...]) with sd varying from 1 to 20 to generate large variation in scales.
- Draws: x_i ∼ N(0,Σ), repeated B = 5000 times.
- Performance metric: Percentage Relative Improvement in Average Loss (PRIAL):
  - PRIAL(Ŝ) := (1 − [∑_{b=1}^B ∥Ŝ^{(b)} − Σ∥^2] / [∑_{b=1}^B ∥S^{(b)} − Σ∥^2]) × 100.
  - PRIAL = 100 when Ŝ coincides with Σ; PRIAL = 0 when Ŝ as poor as S; negative when worse than S.
- Methods compared (known mean setting):
  - LW (Ledoit and Wolf (2004)):
    - S_LW := (1−ρ_LW) S + ρ_LW tr(S)/p I,
    - ρ_LW := min{ [∑_{i=1}^n ∥x_i x_i^T − S∥^2] / [n^2 (tr(S^2) − tr(S)^2 / p)], 1 }.
  - OAS (Chen et al. (2009)):
    - S_OAS := (1−ρ_OAS) S + ρ_OAS tr(S)/p I,
    - ρ_OAS := min{ [1 − 2/p] [tr(S^2) + tr(S)^2] / [ (n+1 − 2/p)(tr(S^2) − tr(S)^2 / p) ], 1 } (modified formula; paper notes a typo in original).
  - SS (Schäfer and Strimmer (2005)):
    - S_SS := (1−ρ_SS) S + ρ_SS diag(S),
    - ρ_SS := min( [∑_{k≠l} dVar(r_{kl})] / [∑_{k≠l} r_{kl}^2], 1 ), where r_{kl} are sample correlations.
  - OD sample analogue: S_OD = Ŝ(ρ_OD(S)).
- Comparison varies parameters {n, sd, γ}, holding medians fixed when varying one parameter at a time.

### Main simulation results (Section 3.2)
- Overall: OASD (S_OASD) tends to produce higher PRIAL and a ρ that tracks the infeasible Oracle ρ_OD(Σ) more closely across variations in sample size n, scale variation sd, and correlation sparsity γ.

- Variation in scales (sd)
  - For most sd (except very small values), OASD exhibits higher PRIAL than average-variance target methods LW and OAS.
  - When sd is small, LW and OAS perform better (OASD is complementary rather than universally dominant).
  - OASD shows ~1 percent higher PRIAL than SS.
  - The sample-analogue OD estimator performs much poorer numerically than other methods in the simulation (difference between formulas with n vs n+1 can numerically change shrinkage substantially; e.g., n = 18 and φ ≈ .065 produces about half the value).
  - All methods show lower PRIAL as sd increases because targets with null off-diagonals approximate Σ worse; correspondingly shrinkage parameters decrease.

- Sparsity of correlation matrix (γ)
  - OASD exhibits higher PRIAL than other methods; improvement relative to LW and OAS can be up to 10 percent when Σ is sparser (γ small).
  - Intuition: as γ → 0 and Σ becomes diagonal, diag(S) target allows shrinking off-diagonals without distorting diagonal elements; average-variance targets face a trade-off.
  - OASD outperforms SS by up to 10 percent; OASD's ρ approximates ρ_OD(Σ) better than ρ_SS.
  - Differences shrink as Σ becomes denser (γ large).

- Sample size (n)
  - OASD performs best across sample sizes n.
  - On average, PRIAL(OASD) is 10 percent higher than LW and OAS and 2 percent higher than SS in the examined region.
  - Differences increase with n in the plotted region but ultimately vanish as n → ∞ as S → Σ.
  - ρ_OASD tracks ρ_OD(Σ) more closely for all n.

### Performance of inverse matrices (Section 3.3)
- In many applications the inverse Σ^{−1} is the ultimate object of interest; shrinkage methods are designed to minimize MSE of Σ, not Σ^{−1}.
- Modified PRIAL for inverses:
  - PRIAL_INV(Ŝ) := (1 − [∑_{b=1}^B ∥{Ŝ^{(b)}}^{−1} − Σ^{−1}∥^2] / [∑_{b=1}^B ∥{S^{(b)}}^+ − Σ^{−1}∥^2]) × 100, where A^+ denotes generalized inverse.
- Simulation findings:
  - The inverse of S_OASD tends to approximate Σ^{−1} better than other methods.
  - Intuition: when Σ is diagonal with disparate diagonal elements (e.g., 1 and 10), diag(S)^{-1} has diagonal entries nearer to Σ^{-1} than [tr(S)/p I]^{-1}.
- Observations:
  - PRIAL_INV for OD is not monotonic in γ and can be higher than both OASD and the Oracle in some regions; in other regions OD can be worse than the generalized inverse of the sample covariance.
  - OD and SS share the same diagonal target but differ in PRIAL_INV because OD’s shrinkage parameter can be much smaller than SS’s, especially when γ is small.

### Summary conclusions drawn in the section
- Targeting diagonal elements via OASD improves covariance estimation relative to average-variance targets when diagonal variation is substantial and when the true correlation matrix is sparse.
- OASD yields better finite-sample approximations to the Oracle shrinkage weight ρ_OD(Σ) across variations in n, sd, and γ.
- OASD also improves approximation to Σ^{−1} in many settings.
- OASD is complementary to existing methods: when diagonal variation is small, average-variance-targeting methods (LW, OAS) may perform better.

*Source: wpiea2023257-print-pdf — section 2.1 Special Case: Known Mean; sections 3, 3.1, 3.2, and 3.3.*

### 3.4  Alternative Method Based on Shrinking Correlation Matrix

### 3.4  Alternative Method Based on Shrinking Correlation Matrix

### Method description
- When the variation in variable scales is large, obtain a sample correlation matrix R, apply existing shrinkage methods to R, and scale back the estimated correlation matrix by multiplying sample standard deviations.
- Existing methods denoted by e = LWcorr, OAScorr, and SScorr yield a shrinkage estimator R^e by shrinking the correlation matrix R toward targets tr(R)/p I and diag(R) respectively, which both equal an identity matrix I.
- The covariance matrix then can be estimated by
  - S^e = diag(S)^{1/2} R^e diag(S)^{1/2}, e ∈ {LWcorr, OAScorr, SScorr}. (Equation (24) in source)
  - diag(S)^{1/2} is a diagonal matrix with the sample standard deviations on the diagonals.

### Observations about the correlation-based approach
- The diagonal elements of S^e equal the sample variances for all estimation methods e ∈ {LWcorr, OAScorr, SScorr}, just like S_OASD, mitigating the disadvantage of the methods with the average variance target, LW and OAS.
- The shrinkage parameter for SScorr remains the same, ρ_SScorr = ρ_SS, since (22) only uses sample correlations, and thus resulting covariance matrices are also identical S_SS = S_SScorr.
- S^e lies in the feasible set of the problem that defines the optimal shrinkage toward the diagonal target (4) for all e ∈ {LWcorr, OAScorr, SScorr} because
  - S^e = diag(S)^{1/2}{(1−ρ_e)R + ρ_e I}diag(S)^{1/2} = (1−ρ_e)S + ρ_e diag(S). (Equation (25) in source)

### Performance and comparison
- OASD still outperforms other methods in terms of MSE.
- Compared to the figures in section 3.2, the average variance methods, LWcorr and OAScorr, exhibit lower MSE when implemented via correlation-matrix shrinkage, but performance still lags behind OASD.
- Intuition: the shrinkage parameter ρ_e may not be optimized to minimize the MSE of the covariance matrix, but ρ_OASD approximates ρ_OD(Σ) that does minimize MSE, so ρ_OASD can generate a lower MSE.
- Figure 5: Comparison of methods through correlation matrix shrinkage
  - Note: The black lines denote the PRIAL for the unfeasible true value Ŝ_OD(Σ).
  - When one parameter is varied, other parameters are fixed at median (n,sd,γ) = (18,10,.5).

### Conclusion (from Section 4 as it follows)
- The paper proposes a novel covariance matrix estimator OASD that achieves a smaller MSE than existing methods when the variation in variable scales is large. It is useful, for example, when different variables have different units.
- Two caveats:
  - Results are based on a normality assumption. Normalization procedures, such as the Box-Cox transformation, may need to be used if the distribution of data deviate substantially from normality.
  - The methods leave the sample variances intact and only shrink the off-diagonal entries. Extensions can allow different levels of shrinkage for diagonal and off-diagonal entries; this is left for future research.

*High-Dimensional Covariance Matrix Estimation: Shrinkage Toward a Diagonal Target — Working Paper No. WP/2023/257*

---


_Source: https://www.imf.org/-/media/files/publications/wp/2023/english/wpiea2023257-print-pdf.pdf_