## _wp1122

## Source details

**Canonical URL:** [_wp1122](https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2011/_wp1122.pdf)

## Other formats

- [Markdown version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2011/_wp1122.pdf.md)
- [Structured JSON version](/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2011/_wp1122.pdf.json)

---

### Key findings and substantive conclusions
- The Brazilian term structure exhibits regime switching behavior; two regimes are identified:
  - Regime 1: high level, slope and volatility (encapsulates the exchange rate crisis from 1998 to 1999 and the presidential election in October 2002).
  - Regime 2: low level, slope, and volatility (characterized by the more recent period of stable monetary and fiscal policy).
- Regimes are highly persistent and regime changes cannot be explained by inflation, GDP growth or other macroeconomic variables.
- All transition probability information is reflected in the slope and the curvature of the yield curve.
- The market price of level risk is relatively high in Regime 2; this is attributable to the relatively low volatility of the term structure factors in that regime.
- The hidden Markov model outperforms a single-regime ATSM benchmark for all estimated measures of fit, including information criteria that penalize for complexity; the improvement originates from both time-series and cross-sectional dimensions, although time-series fit remains relatively poor.
- One-step-ahead forecasting tests were not performed due to the complexity of re-estimating the models.

### Motivation and rationale
- Stationarity assumptions in most term structure models are challenged by documented time variation and non-stationarity in interest rates and macroeconomic variables; regime-switching approaches (Hamilton (1989, 1990)) are motivated.
- Structural changes can have permanent effects on levels and volatilities (examples cited: US Federal Reserve 1979–1982; Brazil stabilization since 2004 and election of Luiz Inácio Lula da Silva).
- Benefits of introducing a hidden Markov process:
  - Mitigates problems from unit roots in volatility (e.g., Bollerslev, Chou and Kroner (1992)).
  - Improves in-sample and out-of-sample fit when regimes change over time (evidence from Gray (1996), Engel and Kim (1996), So, Lam and Li (1998)).
  - Captures deviations from Gaussian/homoscedastic assumptions important for pricing and risk management of fixed income portfolios.

### Model framework and technical structure
- Affine Term Structure Models (ATSMs) of the A0(3) class used; three factors interpret as level, slope and curvature and explain over 90% of yield variance.
- Discrete-time Gaussian homoscedastic ATSM specification (selected equations preserved):
  - r_t = a + b X_t  (Equation (1))
  - X_t = X_{t−1} + K(θ − X_{t−1})∆t + Σ ε_t  (Equation (2))
  - Ψ_t = Σ^{−1}(λ_0 + Λ_1 X_t)  (Equation (3))
- Closed-form bond pricing in single-regime ATSM:
  - P(t,τ) = e^{A(τ) + B(τ) X_t} (Equation (4))
  - Recursions:
    - A(τ+1) = A(τ) + B(τ)(Kθ − λ_0) + 1/2 B(τ)' ΣΣ' B(τ) − a  (Equation (5))
    - B(τ+1) = B(τ) − B(τ)(K − Λ_1) − b  (Equation (6))
- Dai, Singleton, and Yang (2007) hidden Markov specification (conditional on regime s_t):
  - r_t = a(s_t) + b X_t  (Equation (9))
  - X_t = X_{t−1} + K(s_t)(θ(s_t) − X_{t−1})∆t + Σ(s_t) ε_t  (Equation (10))
  - Ψ_t(s_t) = Σ(s_t)^{−1}(λ_0(s_t) + Λ_1(s_t) X_t)  (Equation (11))
- Market price of regime-switching risk parameterization:
  - Γ_{i,j}(t) = log(π^P_{i,j}(t) / π^Q_{i,j}), ∀ i,j  (Equation (12))
- Bond pricing with regimes and recursions preserved (Equations (13)–(15)).
- Two-regime identification conventions: Regime 1 Σ = identity, K lower triangular, θ = 0 vector; Regime 2 Σ diagonal.

### Estimation approach and methodological choices
- Bayesian Markov Chain Monte Carlo (MCMC) estimation strategy chosen over ML to:
  - Generate full posterior distributions and consistent standard errors.
  - Address small sample issues in Brazilian term structure data prior to 1998.
  - Allow direct testing of equality between parameters across regimes.
- MCMC advantages emphasized: posterior densities vs. asymptotic ML standard errors.
- Algorithmic building blocks summarized: Gibbs Sampler and Metropolis-Hastings; full posterior, prior specifications, and full conditionals provided.
- Summary MCMC algorithm (iterated steps):
  1. Generate Ω(s_t) Ω(s_t)' for each state using (40).
  2. Generate Θ via one Metropolis-Hastings step using (34)-(36).
  3. Produce Γ via (37) and obtain Π^P(t) for all t.
  4. Generate S recursively from (38)-(39).
  5. Obtain Π^Q via (26)-(28).
  6. Return to step 1.
- Computational considerations:
  - Over 1 million iterations were run to achieve convergence; a sample of 200,000 was collected.
  - High degree of thinning required; targeted Metropolis acceptance rates between 20 percent and 25 percent.

### Yield specification, factor inversion, and likelihood
- Yield definition and relations:
  - r(t,τ) = − ln(P(s_t, t, τ)) / τ  (equation (16))
  - r(t,τ) = − A(s_t, τ)/τ + B(τ) X_t / τ  (equation (17))
  - Stacked form: R_t = A(s_t) + B X_t  (equation (18))
- Measurement error approaches:
  - All yields matched with error: R_t = A(s_t) + B X_t + ε_t (equation (19)); Kalman Filter estimation.
  - Subset matched without error (n = number of latent factors) used here to preserve arbitrage-free conditions and invert yields:
    - X_t = B̂^{−1} (R̂_t − Â(s_t)) (equation (20))
    - Resulting measurement equation: R̃_t = Ã(s_t) − B̃ B̂^{−1} Â(s_t) + B̃ B̂^{−1} R̂_t + Ω(s_t) e_t (equation (21))
- Likelihood conditional on regime vector S and parameters given as product of Normal densities (equation (23)).

### Data (Brazilian application)
- Term structure data:
  - Instrument: Di-Pre (Deposito Interbanco) swap rates.
  - Swap maturities: 6, 12, 18, 24, 30, and 36 month maturities.
  - Sample period: January 1998 to May 2007 (monthly data).
  - Swaps are single payment at maturity.
- Summary statistics for Brazilian swap rates (Table 1):
  - 6 mth: Mean 20.9433; Median 19.2600; Stdev 6.2510; Skew 1.6984; Kurt 4.5881; Lag 1 0.8706; Lag 2 0.6956
  - 12 mth: Mean 21.6156; Median 19.3800; Stdev 6.7365; Skew 1.3453; Kurt 2.8871; Lag 1 0.8809; Lag 2 0.7315
  - 18 mth: Mean 22.2440; Median 19.5650; Stdev 7.2997; Skew 1.1699; Kurt 1.9330; Lag 1 0.8917; Lag 2 0.7572
  - 24 mth: Mean 22.6781; Median 19.9200; Stdev 7.6969; Skew 1.0636; Kurt 1.3445; Lag 1 0.8974; Lag 2 0.7703
  - 30 mth: Mean 23.0327; Median 20.4800; Stdev 7.9486; Skew 0.9644; Kurt 0.8773; Lag 1 0.8980; Lag 2 0.7710
  - 36 mth: Mean 23.3138; Median 20.9050; Stdev 8.1618; Skew 0.9040; Kurt 0.5818; Lag 1 0.8991; Lag 2 0.7738
- Empirical observations:
  - Long term swap rates more volatile than short rates; long term rates more persistent and have lower kurtosis.
  - Average yield curve over the period is relatively flat.
- Macroeconomic variables for regime transition (M_t):
  - M_t includes a vector of ones and three term structure variables; data limitations restrict adding many macro variables.
  - Selected macro variables: %∆GDP (Reais) and Inflation, both treated as stationary per ADF test p-values.
- Table 2 summary:
  - %∆GDP (Reais): Mean 0.0096; Median 0.0082; Stdev 0.0380; Skew 0.1257; Kurt −0.6497; Lag 1 −0.0267; Lag 2 −0.1622; ADF test p-value 0.0100
  - Inflation: Mean 0.0055; Median 0.0047; Stdev 0.0049; Skew 1.8897; Kurt 6.6624; Lag 1 0.6553; Lag 2 0.3837; ADF test p-value 0.0208

### Empirical results: hidden Markov behavior and regimes
- Models estimated:
  - Single regime ATSM (ATSM model).
  - Two-regime hidden Markov model (HMM model).
- Principal finding: Brazilian term structure exhibits hidden Markov behavior with two regimes:
  - Regime 1: high level, steep slope, high volatility.
  - Regime 2: low level, low slope, low volatility.
- Historical association of Regime 1: end of the Asian crisis, Russian default consequences, Brazilian currency crisis in 1998–1999, April 2001 to August 2003 including October 2002 election.

### Regime identification, persistence, and transition probabilities
- Two regimes with mean durations (half-life):
  - Mean duration of Regime 1 is 8.81 months.
  - Mean duration of Regime 2 is approximately 13.12 months.
- Estimated homogeneous transition probability matrix Π_Q (reported with 95 percent credible intervals):
  - Row 1: 0.9244% 0.0756%  (95 percent credible intervals: (0.8800; 0.9444)   (0.0555; 0.1200))
  - Row 2: 0.0515% 0.9485%  (95 percent credible intervals: (0.0312; 0.0857)   (0.9153; 0.9686))
- Practical implication: once in a regime, the probability of transitioning to the other regime is low.

### Drivers of regime changes and explanatory variables
- Term structure factors drive regime changes; slope and level are primary, curvature is smaller:
  - Term structure factors: curvature = X[1], level = −X[2], slope = −X[3].
- Macroeconomic variables:
  - GDP growth (∆GDP) and inflation coefficients are not significant in explaining regime transitions conditional on term structure factors.
  - Interpretation: macroeconomic information appears to be incorporated into latent term structure factors rather than directly explaining state transitions.
- Parameter estimation issues:
  - Large variances in parameter estimates due to small sample and limited explanatory power of exogenous variables; posterior tests revealed no statistically relevant differences between many parameters.

### Term structure characteristics and model comparison (fit and dynamics)
- Mean reversion (real parts of eigenvalues):
  - Eigen(K_HMM(1)) = [0.3759, 0.1637, 0.1213]
  - Eigen(K_HMM(2)) = [0.3576, 0.0531, 0.0034]
  - Eigen(K_ATSM) = [0.2950, 0.1070, 0.1649]
  - Interpretation: mean reversion in Regime 1 is greater than in Regime 2; comparison with ATSM is inconclusive.
- Factor interpretations: X[2] = level, X[3] = slope, X[1] = curvature (X[1] only moderately correlated to curvature defined by the butterfly).
- Model fit and penalized criteria (Table 6):
  - Effective number of parameters (k): ATSM = 16.7; HMM = 31.4.
  - Fit statistics (Total):
    - ATSM: DIC 1361.781, AIC 1378.471, BIC 1442.181
    - HMM: DIC -891.240, AIC -922.600, BIC -1042.857
  - Mean(log(L)) (Total):
    - ATSM: -672.545
    - HMM: -429.940
  - Time-series vs cross-sectional log likelihoods (Mean ln(L)):
    - ATSM: Time-Series -657.589 ; Cross-Section -14.956
    - HMM: Time-Series -518.085 ; Cross-Section 88.144
  - Credible intervals for HMM ln(L):
    - 97.5%(ln(L)): -445.525 (Total), -528.955 (Time), 76.520 (Cross)
    - 2.5%(ln(L)): -415.835 (Total), -509.334 (Time), 98.698 (Cross)
- Overall: HMM exhibits superior in-sample fit on all considered measures, attributable to both time-series and cross-sectional dimensions; both models fit cross-sectional contemporaneous rates better than time-series dynamics.

### Market prices of risk
- Market price of factor risk:
  - Regime 2 shows more volatile market prices of risk for all factors and a higher price for the level factor.
  - Consistent with risk premium being higher in the low volatility regime.
- Market price of regime switching risk:
  - Calculated via Γ_{i,j}(t) = log(π^P_{i,j}(t) / π^Q_{i,j}).
  - Price is negative when the real-world probability of regime change is lower than the risk-neutral probability.
  - Price of transitioning from high volatility regime to low volatility regime is lower than the reverse.
  - At the end of the sample, the risk of switching to Regime 2 is presumably relatively low.

### Policy implications, applications, and caveats
- Practical uses:
  - Hidden Markov models useful where distributional characteristics matter: risk management, fixed income management, public debt management, and scenario analysis (identifying data-driven state baselines).
  - HMMs are less appropriate for one-step-ahead forecasting and pricing where arbitrage-free models are preferable.
- Forecasting caveats:
  - Estimated regimes are highly persistent, making accurate forecasting of regime changes difficult.
  - Time-series forecasting performance of affine models remains poor; one-step-ahead forecasts are not practical in this framework.
- Specific recommendation for public debt management and policy:
  - Incorporate possibility of more volatile yields (Regime 1 characteristics) into forecasting and policy decisions.
  - Use HMMs for scenario generation and stress testing rather than for operational forecasting or pricing.

### Limitations and directions for future research
- Limitations:
  - Small sample and extreme term structure behavior make parameter estimation noisy.
  - Two regimes may be insufficient to capture extreme early-sample behavior.
  - Overidentifying restrictions (Dai, Singleton and Yang (2007)) may not fully describe Regime 1; relaxing restrictions did not eliminate the non-zero mean issue.
- Suggested future research:
  - Develop restrictions that allow simplified estimation of hidden Markov models.
  - Benchmark forecasting performance against models such as dynamic Nelson and Siegel.
  - Investigate HMMs' ability to generate realistic scenarios for interest rate outcomes and further disentangle regime shifts from direct macro-dependencies.

*Source: _wp1122 - References; Section III; conclusions.*

### References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

### _wp1122 - References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

### Key findings and substantive conclusions
- The Brazilian term structure exhibits regime switching behavior; two regimes are identified:
  - Regime 1: high level, slope and volatility (encapsulates the exchange rate crisis from 1998 to 1999 and the presidential election in October 2002).
  - Regime 2: low level, slope, and volatility (characterized by the more recent period of stable monetary and fiscal policy).
- Regimes are highly persistent and regime changes cannot be explained by inflation, GDP growth or other macroeconomic variables.
- All transition probability information is reflected in the slope and the curvature of the yield curve.
- The market price of level risk is relatively high in Regime 2; this is attributable to the relatively low volatility of the term structure factors in that regime.
- The hidden Markov model outperforms a single-regime ATSM benchmark for all estimated measures of fit, including information criteria that penalize for complexity; the improvement originates from both time-series and cross-sectional dimensions, although time-series fit remains relatively poor.
- One-step-ahead forecasting tests were not performed due to the complexity of re-estimating the models.

### Motivation and rationale
- Most term structure models assume stationarity; documented time variation and non-stationarity in interest rates and macroeconomic variables motivate regime-switching approaches (Hamilton (1989, 1990) literature cited).
- Structural changes (sudden or gradual) can have permanent effects on levels and volatilities (examples: US Federal Reserve 1979–1982; Brazil stabilization since 2004 and election of Luiz Inácio Lula da Silva).
- Benefits of introducing a hidden Markov (regime) process:
  - Mitigates problems from unit roots in volatility (e.g., Bollerslev, Chou and Kroner (1992)).
  - Improves in-sample and out-of-sample fit when regimes change over time (evidence from Gray (1996), Engel and Kim (1996), So, Lam and Li (1998)).
  - Captures deviations from usual Gaussian/homoscedastic assumptions important for pricing and risk management of fixed income portfolios.

### Model framework and technical structure
- Affine Term Structure Models (ATSMs) of the A0(3) class are the relevant subset for the DSY hidden Markov model.
- Three factors are used because they explain over 90% of yield variance and have interpretation as level, slope and curvature.
- Discrete-time Gaussian homoscedastic ATSM specification (as presented):
  - r_t = a + b X_t  (Equation (1))
  - X_t = X_{t−1} + K(θ − X_{t−1})∆t + Σ ε_t  (Equation (2))
  - Ψ_t = Σ^{−1}(λ_0 + Λ_1 X_t)  (Equation (3))
- Closed-form bond pricing in the single-regime ATSM: P(t,τ) = e^{A(τ) + B(τ) X_t} (Equation (4)), with recursion:
  - A(τ+1) = A(τ) + B(τ)(Kθ − λ_0) + 1/2 B(τ)' ΣΣ' B(τ) − a  (Equation (5))
  - B(τ+1) = B(τ) − B(τ)(K − Λ_1) − b  (Equation (6))
- Normalization/identification for the A0(3) model: specified structure for K, θ and Σ with K diagonal elements positive to ensure stationarity; example normalization matrices provided in the text.
- Dai, Singleton, and Yang (2007) hidden Markov specification (conditional on regime s_t):
  - r_t = a(s_t) + b X_t  (Equation (9))
  - X_t = X_{t−1} + K(s_t)(θ(s_t) − X_{t−1})∆t + Σ(s_t) ε_t  (Equation (10))
  - Ψ_t(s_t) = Σ(s_t)^{−1}(λ_0(s_t) + Λ_1(s_t) X_t)  (Equation (11))
- Under the risk-neutral measure Q, transition probabilities must be homogeneous; the market price of regime-switching risk is parameterized by:
  - Γ_{i,j}(t) = log(π^P_{i,j}(t) / π^Q_{i,j}), ∀ i,j  (Equation (12))
- Bond pricing with regimes: P(s_t,t,τ) = e^{A(s_t,τ) + B(τ) X_t} (Equation (13)), with recursions:
  - A(s_t,τ+1) = B(τ)(K(s_t) θ(s_t) − λ_0(s_t)) + log(Σ_{j=0}^S π^Q_{i,j} e^{A(j,τ)}) + 1/2 B(τ) Σ(s_t) Σ(s_t)' B(τ)' − a(s_t)  (Equation (14))
  - B(τ+1) = −b + B(τ)(1 − K(s_t) − Λ_1(s_t))  (Equation (15))
- Identification in the two-regime case: DSY assume Regime 1 Σ is identity, K is lower triangular and θ is a vector of zeros; Regime 2 Σ is diagonal to ensure factor independence.

### Estimation approach and methodological choices
- The paper adopts a Bayesian Markov Chain Monte Carlo (MCMC) estimation strategy instead of the ML method used by DSY, to:
  - Generate full posterior distributions of parameters and obtain consistent standard error estimates.
  - Address small sample issues present in Brazilian term structure data prior to 1998.
  - Allow direct testing of equality between parameters across regimes.
- MCMC advantages emphasized: posterior densities vs. asymptotic standard errors from ML (asymptotic errors may differ substantially in short samples).
- The paper provides a step-by-step guideline to applying the Bayesian MCMC algorithm and summarizes Gibbs and Metropolis-Hastings algorithms (Boxes: 1 Gibbs Sampler, 2 Metropolis-Hastings Algorithm).
- Data likelihood links the arbitrage-free closed-form solution (expressions (14) and (15)) to observable yields; focus is on specifying the likelihood for estimation.

### Model limitations and caveats
- Affine structure limitations: ATSMs share shortcomings including difficulty of estimation due to non-linear arbitrage-free restrictions; positivity of yields not guaranteed.
- Number of regimes is not known a priori; model comparison is difficult because models with different numbers of regimes are not nested.
- Regimes may or may not be predictable, which can adversely affect forecasting performance.
- The restriction requiring homogeneous Π^Q under Q is acknowledged as counterfactual but noted to have small effect on model fit because estimation is carried out under P and the restriction mainly affects cross-sectional fit.

*Source: _wp1122 - References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .*

### Section III.

### _wp1122 - Section III

### Yield specification and factor inversion
- Yield at time t with maturity τ denoted by r(t,τ) is defined as
  - r(t,τ) = − ln(P(s_t, t, τ)) / τ (equation (16))
  - With P(s_t, t, τ) given by expression (13), this implies
    - r(t,τ) = − A(s_t, τ)/τ + B(τ) X_t / τ (equation (17))
- Stacking yields into an (m×1) vector R_t and coefficients into A(s_t) and B yields
  - R_t = A(s_t) + B X_t (equation (18))
- Assuming a three-dimensional latent factor vector X_t (standard in ATSMs), A(s_t,τ)/τ has dimension (m×1) and B(τ)/τ has dimension (m×3).
- Two approaches to measurement error:
  - All yields matched with error:
    - R_t = A(s_t) + B X_t + ε_t (equation (19)); model estimated using a Kalman Filter.
  - Only a subset (m−n) yields matched with error and n yields modeled without error (n = number of latent factors X_t). Advantage: term structure arbitrage-free at n maturities and latent factors X_t obtained by inverting yields; simplifies estimation and is employed here.
- Inversion for factors using yields matched without error:
  - X_t = B̂^{−1} (R̂_t − Â(s_t)) (equation (20))
- Yields matched with error follow:
  - R̃_t = Ã(s_t) − B̃ B̂^{−1} Â(s_t) + B̃ B̂^{−1} R̂_t + Ω(s_t) e_t (equation (21))
  - Ω(s_t) is the Cholesky decomposition of the error covariance matrix and e_t is a standard normal error vector.
- Expression for R̂_t incorporating dynamics of X_t:
  - R̂_t = μ̂_R + B̂ Σ(s_t) ε_t (equation (22))
  - μ_t = Â(s_t) + B̂ (K(s_t) θ(s_t) − (1−K(s_t)) B̂^{−1} Â(s_{t−1})) + B̂ (1−K(s_t)) B̂^{−1} R̂_{t−1}
- Likelihood conditional on regime vector S = {s_1,...,s_T} and model parameters:
  - L(R|S,...) = ∏_{t=1}^T N(R̃ mean, Ω(s_t)Ω(s_t)' ) × N(μ_t, B̂ Σ(s_t) Σ(s_t)' B̂' ) (equation (23)), where R is a matrix of stacked yields.

### The state process (S) and transition probabilities
- Under the real-world measure P:
  - Each s_t ∼ Categorical(Π^P_{s_t}(t)) with Π^P(t) heterogeneous over time (equation (24)).
  - Elements π^P_{i,j}(t) depend on a linear index through a Logit transformation:
    - Logit(π^P_{s_t,s_{t+1}}(t)) = γ_{s_t,s_{t+1}} M_t (equation (25))
    - M_t is a (k×1) vector of known macroeconomic or term structure variables.
    - Only q−1 probabilities need estimation per row of Π^P(t); stack γ into Γ of dimension (k q (q−1) × 1).
- Under the risk-neutral measure Q:
  - s_t evolves with a homogeneous probability matrix Π^Q (equation (26)).
  - Π^Q_i distributed as Dirichlet(α_{s_t,1},...,α_{s_t,q}) (equation (27)), with
    - α_{s_t,s_{t+1}} = (1/T) ∑_{t=1}^T 1(s_t = i, s_{t+1} = j) (equation (28)).

### Bayesian estimation: posterior, priors, and MCMC algorithm
- Full posterior:
  - p(Θ, S, Ω, Γ | R) ∝ p(R | Θ, S, Ω, Γ) p(Θ, S, Ω, Γ) (equation (29)), where Θ = {a(s_t), b, K(s_t), θ(s_t), Σ(s_t), λ_0(s_t), Λ_1(s_t)}.
  - Prior factorization: p(Θ, S, Ω, Γ) ∝ p(Θ | S, Ω, Γ) p(Ω | S, Γ) p(S | Γ) p(Γ) (equation (30)).
- Prior specifications:
  - p(Θ | S, Ω, Γ) = N(μ_Θ, σ_Θ) (diffuse multivariate normal) (equation (31)).
  - p(Ω Ω' | S, Ω, Γ) = InvWishart(G, g) (equation (32)).
  - p(Γ) = N(μ_Γ, σ_Γ) (equation (33)).
- Full conditionals and sampling:
  - Full conditional for Θ:
    - p(Θ | R, S, Ω, Γ) ∝ ∏_{t=1}^T p(R̂_t | Θ, S, Ω, Γ) p(R̃_t | Θ, S, Ω, Γ) p(Θ) (equation (34)),
    - p(R̂_t | ...) = N(μ_{R̂}, B̂ Σ(s_t) Σ(s_t)' B̂') (equation (35)),
    - p(R̃_t | ...) = N(Ã(s_t) − B̃ B̂^{−1} Â(s_t) + B̃ B̂^{−1} R̂_t, Ω(s_t) Ω(s_t)') (equation (36)).
    - Kernel of p(Θ | ...) is non-recognizable; Metropolis-Hastings used within Gibbs.
  - Full conditional for Γ:
    - p(Γ | S, Γ, M) ∝ ∏_{i=1}^q ∏_{j=1}^q ∏_{t=1}^T ( e^{γ_{i,j} M_t} / ∑_{S_i=1}^q e^{γ_{i,j} M_t} )^{1(s_t=j, s_{t−1}=i)} p(Γ) (equation (37)).
    - Kernel unknown due to logistic structure; Metropolis-Hastings used.
  - Full conditional for each s_t sampled recursively from categorical with parameter vector h:
    - p(s_t = i | s_{t−1}, s_{t+1}, Θ, Ω, R_t, Π^P(t), Π^P(t+1)) normalized across j = 1..q (equation (38)),
    - where p(s_t = i | ...) = π^P_{s_{t−1}, s_t}(t) p(R̃_t | R̂_t, Θ, s_t, s_{t−1}) p(R̂_t | R̂_{t−1}, Θ, s_t) π^P_{s_t, s_{t+1}}(t) (equation (39)).
  - Full conditional for Ω(s_t) Ω(s_t)' is Inverse Wishart:
    - (Ω(i) Ω(i)' | R, Θ, S) ∼ InvWishart( G + ∑_{t=1}^T 1(s_t = i) E' E, k + T_i ) (equation (40)),
    - E = vector of stacked errors e_t = 1(s_t = i) (R_t − Ã(s_t) − B̃ X_t), T_i = ∑_{t=1}^T 1(s_t = i); Ω(s_t) obtained by Cholesky decomposition of Ω(s_t) Ω(s_t)'.
- Metropolis-Hastings and Gibbs:
  - Box 1 outlines the Gibbs sampler for joint sampling via full conditionals.
  - Box 2 outlines the Metropolis-Hastings algorithm used when kernel forms are not recognized or for proposal/acceptance steps.
- Summary MCMC algorithm steps (iterated):
  1. Generate Ω(s_t) Ω(s_t)' for each state using (40).
  2. Generate Θ via one Metropolis-Hastings step using (34)-(36).
  3. Produce Γ via (37) and obtain Π^P(t) for all t.
  4. Generate S recursively from (38)-(39).
  5. Obtain Π^Q via (26)-(28).
  6. Return to step 1.
- After convergence, discard burn-in and use draws from joint posterior; required number of iterations depends on autocorrelation/mixing.

### Data (Brazilian application)
- Term structure data:
  - Brazilian term structure characterized by Di-Pre (Deposito Interbanco) swap rates.
  - Swap maturities: 6, 12, 18, 24, 30, and 36 month maturities.
  - Sample period: January 1998 to May 2007 (monthly data).
  - Swaps available monthly and are single payment at maturity.
- Table 1. Brazilian Swap Rates Summary (central moments, median, autocorrelations)
  - 6 mth:
    - Mean 20.9433; Median 19.2600; Stdev 6.2510; Skew 1.6984; Kurt 4.5881; Lag 1 0.8706; Lag 2 0.6956
  - 12 mth:
    - Mean 21.6156; Median 19.3800; Stdev 6.7365; Skew 1.3453; Kurt 2.8871; Lag 1 0.8809; Lag 2 0.7315
  - 18 mth:
    - Mean 22.2440; Median 19.5650; Stdev 7.2997; Skew 1.1699; Kurt 1.9330; Lag 1 0.8917; Lag 2 0.7572
  - 24 mth:
    - Mean 22.6781; Median 19.9200; Stdev 7.6969; Skew 1.0636; Kurt 1.3445; Lag 1 0.8974; Lag 2 0.7703
  - 30 mth:
    - Mean 23.0327; Median 20.4800; Stdev 7.9486; Skew 0.9644; Kurt 0.8773; Lag 1 0.8980; Lag 2 0.7710
  - 36 mth:
    - Mean 23.3138; Median 20.9050; Stdev 8.1618; Skew 0.9040; Kurt 0.5818; Lag 1 0.8991; Lag 2 0.7738
- Empirical observations:
  - Long term swap rates more volatile than short rates (possibly due to liquidity distortions at long end).
  - Long term rates are more persistent and have lower kurtosis.
  - Average yield curve over the period is relatively flat.
- Macroeconomic determinants and variables used for regime transitions:
  - M_t includes a vector of ones and three term structure variables; data limitations (114 monthly observations) restrict inclusion of many macro variables.
  - For two-regime model, Γ dimension is eight before macro variables. Selected macro variables added to explain regime changes: GDP growth and inflation.
- Table 2. Macroeconomic Variables Summary (central moments, median, autocorrelations, ADF test p-values)
  - %∆GDP (Reais):
    - Mean 0.0096; Median 0.0082; Stdev 0.0380; Skew 0.1257; Kurt −0.6497; Lag 1 −0.0267; Lag 2 −0.1622; ADF test p-value 0.0100
  - Inflation:
    - Mean 0.0055; Median 0.0047; Stdev 0.0049; Skew 1.8897; Kurt 6.6624; Lag 1 0.6553; Lag 2 0.3837; ADF test p-value 0.0208
  - Both inflation and GDP are reported as stationary according to Dickey-Fuller p-values.
  - Both inflation and GDP obtained from the IFS database.

### Empirical results: Hidden Markov behavior and regimes
- Models estimated:
  - Single regime ATSM (denoted ATSM model) for comparison.
  - Two-regime hidden Markov model (denoted HMM model) as detailed in Section III.
- Principal finding:
  - Brazilian term structure exhibits hidden Markov behavior.
  - Two regimes identified:
    - Regime 1: high level, slope and volatility regime.
    - Regime 2: low level, slope and volatility regime.
- Regime characterization and historical association:
  - Regime 1 captures:
    - End of the Asian crisis,
    - Consequences of Russian default,
    - Brazilian currency crisis in 1998 and 1999,
    - Period from April 2001 to August 2003 encompassing the Brazilian election in October 2002.
  - Regime 2 identified from early [text truncated in source at this point].

*Source: _wp1122 - Section III.*

### conclusions.

### conclusions.

### Regime identification and persistence
- Two regimes identified in the Brazilian term structure:
  - Regime 1: high level, steep slope, high volatility, higher mean reversion, higher factor and error variances.
  - Regime 2: low level, low slope, low volatility; associated with stable monetary and fiscal policy in recent years.
- Transition dynamics and persistence:
  - Mean duration (half-life) of Regime 1 is 8.81 months.
  - Mean duration (half-life) of Regime 2 is approximately 13.12 months.
  - Estimated homogeneous transition probability matrix Π_Q:
    - Row 1: 0.9244% 0.0756%  (95 percent credible intervals: (0.8800; 0.9444)   (0.0555; 0.1200))
    - Row 2: 0.0515% 0.9485%  (95 percent credible intervals: (0.0312; 0.0857)   (0.9153; 0.9686))
  - Practical implication: once in a regime, the probability of transitioning to the other regime is low.

### Drivers of regime changes and explanatory variables
- Regime changes are dependent on the level and slope of the term structure; curvature plays a smaller role.
  - Term structure factors: curvature = X[1], level = −X[2], slope = −X[3].
- Macroeconomic variables:
  - GDP growth (∆GDP) and inflation coefficients are not significant in explaining regime transitions conditional on term structure factors.
  - Interpretation: macroeconomic information appears to be incorporated into latent term structure factors rather than directly explaining state transitions.
- Parameter estimation issues:
  - Large variances in parameter estimates due to small sample and limited explanatory power of exogenous variables; posterior tests revealed no statistically relevant differences between many parameters.

### Term structure characteristics and model comparison
- Model types compared: single-regime ATSM and hidden Markov HMM (bxHMM).
- Mean reversion (real parts of eigenvalues) benchmarks:
  - Eigen(K_HMM(1)) = [0.3759, 0.1637, 0.1213]
  - Eigen(K_HMM(2)) = [0.3576, 0.0531, 0.0034]
  - Eigen(K_ATSM) = [0.2950, 0.1070, 0.1649]
  - Interpretation: mean reversion in Regime 1 is greater than in Regime 2; comparison with ATSM is inconclusive.
- Factor interpretations:
  - X[2] = level, X[3] = slope, X[1] = curvature (X[1] only moderately correlated to curvature defined by the butterfly).
  - ATSM and HMM factors are highly correlated; the most conspicuous disparity is the level (X[2] and X[3]).
- Model fit and penalized criteria (Table 6):
  - Effective number of parameters (k): ATSM = 16.7; HMM = 31.4.
  - Fit statistics (Total):
    - ATSM: DIC 1361.781, AIC 1378.471, BIC 1442.181
    - HMM: DIC -891.240, AIC -922.600, BIC -1042.857
  - Mean(log(L)) (Total):
    - ATSM: -672.545
    - HMM: -429.940
  - Time-series vs cross-sectional log likelihoods (Mean ln(L)):
    - ATSM: Time-Series -657.589 ; Cross-Section -14.956
    - HMM: Time-Series -518.085 ; Cross-Section 88.144
  - Credible intervals for log likelihood:
    - HMM 97.5%(ln(L)): -445.525 (Total), -528.955 (Time), 76.520 (Cross)
    - HMM 2.5%(ln(L)): -415.835 (Total), -509.334 (Time), 98.698 (Cross)
- Overall: HMM exhibits superior in-sample fit on all considered measures (including complexity-penalized criteria). Improvement is attributable to both time-series and cross-sectional dimensions. Both models fit cross-sectional contemporaneous rates better than time-series dynamics.

### Market prices of risk
- Market price of factor risk:
  - Regime 2 shows more volatile market prices of risk for all factors and a higher price for the level factor.
  - Consistent with the finding that risk premium can be higher in the low volatility regime.
- Market price of regime switching risk (calculated via expression (12)):
  - The price is negative when the real-world probability of regime change is lower than the risk-neutral probability.
  - Price of transitioning from high volatility regime to low volatility regime is lower than the reverse.
  - At the end of the sample, the risk of switching to Regime 2 is presumably relatively low.

### Estimation, convergence, and computational considerations
- Bayesian MCMC algorithm (Metropolis-Hastings within Gibbs) developed for HMM estimation; algorithm produces consistent standard errors.
- Convergence challenges:
  - Small sample size and highly non-linear ATSMs lead to slow mixing and high autocorrelation in posterior samples.
  - Over 1 million iterations were run to achieve convergence; a sample of 200,000 was collected.
  - High degree of thinning required to obtain a reliable sample.
- RW Metropolis-Hastings implementation details:
  - Proposal covariance obtained from covariances of initial sample based on a naïve covariance matrix.
  - Targeted acceptance rates between 20 percent and 25 percent (Roberts and Rosendal guidance around 23.4 percent).

### Policy implications, applications, and caveats
- Practical uses:
  - Hidden Markov models are useful where distributional characteristics matter: risk management, fixed income management, public debt management, and scenario analysis (identifying data-driven state baselines).
  - HMMs are less appropriate for one-step-ahead forecasting and pricing where arbitrage-free models are preferable.
- Forecasting caveats:
  - Estimated regimes are highly persistent, making accurate forecasting of regime changes difficult.
  - Time-series forecasting performance of affine models remains poor; one-step-ahead forecasts are not practical in this framework.
- Specific recommendation for public debt management and policy:
  - Incorporate possibility of more volatile yields (Regime 1 characteristics) into forecasting and policy decisions.
  - Use HMMs for scenario generation and stress testing rather than for operational forecasting or pricing.

### Limitations and directions for future research
- Limitations highlighted:
  - Small sample and extreme term structure behavior make parameter estimation noisy.
  - Two regimes may be insufficient to capture extreme early-sample behavior.
  - Overidentifying restrictions (Dai, Singleton and Yang (2007)) may not fully describe Regime 1; relaxing restrictions did not eliminate the non-zero mean issue.
- Suggested future research:
  - Develop restrictions that allow simplified estimation of hidden Markov models.
  - Benchmark forecasting performance against more popular models such as dynamic Nelson and Siegel.
  - Investigate the ability of HMMs to generate realistic scenarios for interest rate outcomes and further disentangle regime shifts from direct macro-dependencies.

*Source: Fund staff estimates; conclusions and numerical results as presented in the chapter "conclusions."*

### REFERENCES

### _wp1122 - REFERENCES

### Term structure and yield curve modeling
- Ang, A. and M. Piazzesi, 2003, ”A no-arbitrage vector autoregression of the term structure dynamics with macroeconomic and latent variables,”Journal of Monetary Economics, 50, 745-787.
- Bansal, R., and H. Zhou, 2002, ”Term structure of interest rates with regime shifts,”Journal of Finance, 57, 1997-2043.
- Bansal, R., G. Tauchen and H. Zhou, 2004, ”Regime-shifts, risk premiums in the term structure, and the business cycle,”Journal of Business and Economic Statistics, 22, 396-409.
- Dai, Q., J. Singleton and W. Yang, 2007, ”Regime shifts in a dynamic term structure model of US treasury bond yields,”Review of Financial Studies, 20, 1669-1706.
- Diebold, F. X., G. D. Rudebusch and B. Aruoba, 2006, ”The macroeconomy and the yield curve: A dynamic latent factor approach,”Journal of Econometrics, 131, 309-338.
- Landen, C., 2000, ”Bond pricing in a hidden Markov model of the short rate,”Finance and Stochastics, 4, 371-389.
- Nelson, C. R. and A. F. Siegel, 1987, ”Parsimonious modeling of yield curves,”The Journal of Business, 60, 473-489.
- Wu, S. and Y. Zeng, 2003, ”Affine regime-switching models for interest rate term structure,” Contemporary Mathematics, 351, 375-386.

### Regime-switching, Markov-switching, and heteroscedasticity
- Gray, S. F., 1996, ”Modeling the conditional distribution of interest rates as a regime-switching process,”Journal of Financial Economics, 42, 27-62.
- Hamilton, J. D., 1989, ”A new approach to the economic analysis of non-stationary time series and the business cycle,”Econometrica, 57, 357-384.
- Hamilton, J. D., 1990, ”Analysis of time series subject to changes in regime,”Journal of Econometrics, 45, 39-70.
- Kim, C., 1994, ”Unobserved component time series models with Markov-switching heteroscedasticity: Changes in regime and the link between inflation rates and inflation uncertainty,”Journal of Business and Economic Statistics, 11, 341-349.
- Kim, C., 1996, ”Predicting business cycle phases with indexes of leading and coincident indicators: A multivariate regime-shift approach,”Journal of Monetary Economics, 12, 1-27.
- So, M. K. P., K. Lam and W. K. Li, 1998, ”A stochastic volatility model with Markov switching,”Journal of Business and Economic Statistics, 16, 244-253.

### Time series, exchange rates, and business cycle analysis
- Engel C. and C. Kim, 1996, ”The long run US/UK real exchange rate,” Working Paper, National Bureau of Economic Research.
- Hamilton, J. D., 1989, ”A new approach to the economic analysis of non-stationary time series and the business cycle,”Econometrica, 57, 357-384.
- Kim, C., 1996, ”Predicting business cycle phases with indexes of leading and coincident indicators: A multivariate regime-shift approach,”Journal of Monetary Economics, 12, 1-27.
- Bansal, R., G. Tauchen and H. Zhou, 2004, ”Regime-shifts, risk premiums in the term structure, and the business cycle,”Journal of Business and Economic Statistics, 22, 396-409.

### Methodology and computational statistics
- Robert, C. P. and G. Casella, 2004, ”Monte Carlo Statistical Methods,” Springer Texts in Statistics.

*Source: _wp1122 - REFERENCES*

---


_Source: https://www.imf.org/-/media/websites/imf/imported-full-text-pdf/external/pubs/ft/wp/2011/_wp1122.pdf_