panelmed
v0.1.1MIT licencePython ≥ 3.10

panelmed

Causal mediation analysis for panel data with double/debiased machine learning: direct, indirect and total effects under two-way fixed effects.

Developed by Dr Merwan Roudane

Implements Wu, P. and Xiao, Z. (2026), “Causal Mediation Analysis for Panel Data with Double/Debiased Machine Learning”, arXiv:2610.21339.

pip install panelmed
Overview

What it does

A treatment often changes an outcome through an intermediate variable. Employment raises local income partly because it raises wages. panelmed splits the total effect into the part that goes through the mediator and the part that does not, using repeated observations of the same units to remove any time-invariant confounder.

Estimator

DML-GMM for the outcome and mediator equations, with cross-fitting and an auxiliary fold that removes period effects.

Effects and inference

Controlled direct, natural direct, natural indirect and total effects, in levels and as derivatives, with analytical and bootstrap standard errors.

Tables and figures

Text, LaTeX, Markdown and HTML tables; decomposition, profile, dose-response, surface and diagnostic figures.

Data included

The U.S. county panel of the paper's application (3,058 counties, 2000–2019) and the paper's Monte Carlo generator.

Setup

Installation

pip install panelmed

With interactive 3-D surfaces:

pip install "panelmed[plotly]"

Requirements: Python 3.10 or later, with numpy, scipy, pandas, matplotlib, scikit-learn and joblib (installed automatically).

Never used Python? Install Anaconda, open “Anaconda Prompt”, and type the command above. The README has a step-by-step guide from zero.

Method

Theory

The model

For unit \(i = 1,\dots,N\) and period \(t = 1,\dots,T\), with outcome \(Y_{it}\), mediator \(M_{it}\), treatment \(D_{it}\) and covariates \(X_{it}\):

$$Y_{it} = \tau_Y(D_{i,t-L:t}, M_{i,t-L:t}, X_{i,t-L:t})'\theta_0 + f_0(X_{i,t-L:t}) + \alpha_i^Y + \lambda_t^Y + \varepsilon_{it}^Y$$ $$M_{it} = \tau_M(D_{i,t-L:t}, X_{i,t-L:t})'\gamma_0 + g_0(X_{i,t-L:t}) + \alpha_i^M + \lambda_t^M + \varepsilon_{it}^M$$

\(\tau_Y\) and \(\tau_M\) are dictionaries chosen by the researcher (current and lagged treatment, squares, interactions). \(f_0\) and \(g_0\) are unknown, possibly nonlinear functions of the covariates. \(\alpha_i\) are unit effects and \(\lambda_t\) period effects. The treatment may be binary, multivalued or continuous; the mediator is continuous.

The effects

For two treatment histories \(\bar d_t\) and \(\bar d_t^{*}\), with \(\bar M_{it}(\cdot)\) the mediator history a unit would have under a given treatment history:

$$\mathrm{NDE}_t = E\big[Y_{it}(\bar d_t, \bar M_{it}(\bar d_t^{*})) - Y_{it}(\bar d_t^{*}, \bar M_{it}(\bar d_t^{*}))\big]$$ $$\mathrm{NIE}_t = E\big[Y_{it}(\bar d_t, \bar M_{it}(\bar d_t)) - Y_{it}(\bar d_t, \bar M_{it}(\bar d_t^{*}))\big]$$ $$\mathrm{TE}_t = \mathrm{NDE}_t + \mathrm{NIE}_t$$

The controlled direct effect fixes the mediator history at a chosen value. For continuous treatments the same objects exist as derivatives with respect to one period's treatment (NDD, NID, TD).

Assumptions

  • No interference between units, no anticipation, consistency (Assumptions 1–3).
  • Sequential conditional independence given the observed history and the unit effects (Assumption 4). Any time-invariant confounder is allowed.
  • Units are independent; within a unit, any serial dependence is allowed. \(N \to \infty\) with \(T \ge 3\) fixed.

The estimator

First differences remove \(\alpha_i\). Units are split into \(K\) folds. For the fold \(k\) where the score is evaluated, a second fold \(k'\) supplies cross-sectional means and the remaining folds train the nuisance functions. Period effects are removed by

$$D_k Q_{it} = (Q_{it} - Q_{i,t-1}) - \frac{1}{|F_{k'}|}\sum_{j \in F_{k'}} (Q_{jt} - Q_{j,t-1}).$$

With residuals \(U^Y_{it}(\theta)\), \(U^M_{it}(\gamma)\) and \(U^D_{it}\), the moments use lagged residuals as instruments, because under sequential exogeneity contemporaneous ones are invalid after differencing:

$$\psi^Y_{it} = A^Y_{it}\,U^Y_{it}(\theta), \qquad A^Y_{it} = \big(U^D_{i2},\dots,U^D_{i,t-1},\,U^M_{i2}(\tilde\gamma),\dots,U^M_{i,t-1}(\tilde\gamma)\big)'$$ $$\psi^M_{it} = A^M_{it}\,U^M_{it}(\gamma), \qquad A^M_{it} = \big(U^D_{i2},\dots,U^D_{i,t-1}\big)'$$

\(\tilde\gamma\) is a preliminary copy of \(\gamma\) learned outside the evaluation fold, which keeps the score linear in the parameters. The two equations are stacked and solved by GMM in closed form. The score is Neyman orthogonal in every nuisance component.

Inference

The stacked estimator is asymptotically normal with a sandwich covariance. Each effect \(\vartheta\) is an average of unit contributions \(\ell(W_i;\beta)\), with influence function

$$\varphi_{\vartheta,i} = \ell(W_i;\beta_0) - \vartheta_0 + J_\vartheta\,\varphi_{\beta,i},$$

which keeps both the sampling variation of the unit contributions and the estimation error of the coefficients. The covariance of effects that add up exactly is singular; Wald tests use a generalised inverse.

Step → equation map

Every block of code is tied to a numbered equation of the paper. The full map is in docs/methods.md.

CodePaperWhat it computes
Dictionary, make_envSection 2.3\(\tau_Y\), \(\tau_M\) at observed or counterfactual histories
polynomial_basisApp. D.1, E.1monomial library for \(f_0, g_0, h_0\)
_diffEq. (7)first differences
fold loop in fit_onceAlgorithm 1, steps 1–2evaluation, auxiliary and training folds
_first_stage, _lassoApp. D.1l1-penalised IV fit with \(\lambda = \sqrt{2\log(2p)/n}\)
star(...)Eq. (8)auxiliary-fold demeaning
instrument loopAlgorithm 1, step 3\(A^Y_{it}\), \(A^M_{it}\) from lagged residuals
solveEq. (12)closed-form stacked GMM
V = A Psi A'Eq. (A.15)sandwich covariance
Counterfactual.nestedEqs. (13), (14), (17)counterfactual mediator and outcome
effect_inferenceEqs. (18)–(21)effects and their influence functions
bootstrapSection 4cross-sectional bootstrap of the whole procedure

Check against the paper

Monte Carlo design of the paper's Table 1 (\(N = 1600\), \(T = 10\)); 300 replications here, 1,000 in the paper.

EstimandBiasSDCoveragePaper: biasPaper: SDPaper: coverage
\(\theta_{D_t}\)-0.00260.02830.947-0.00190.02740.951
\(\gamma_{D_t}\)-0.00830.03420.970-0.00930.03420.955
NDE0.00410.07460.9300.00150.07310.950
NIE-0.02740.09850.943-0.02740.09930.944
TE-0.02330.10440.977-0.02590.10530.947
Naive-GMM, \(\gamma_{D_t}\)-0.20040.02650.000-0.17700.02710.000
Reference

API and syntax

PanelMediation(y, m, d, x=None, *, outcome, mediator, ...)

ArgumentDefaultMeaning
y, m, drequiredoutcome, mediator, treatment: column names with data, or (N, T) arrays
xNonecovariates: list of column names or (N, T, K) array
outcomerequiredoutcome dictionary \(\tau_Y\) (Section 2.3)
mediatorrequiredmediator dictionary \(\tau_M\) (Section 2.3)
data, unit, timeNonelong-format balanced panel and its identifier columns
basis_degree3degree of the polynomial library (App. D.1)
basis_max_varsNoneat most this many distinct covariates per monomial (App. E.1)
iv_interact"auto"covariates interacted with the lagged treatment residual in the instruments
iv_square"auto"add the squared lagged treatment residual as an instrument

Dictionary terms are expressions in d, m, the covariate names and the lag prefix L1_, L2_, …: "d", "d**2", "L1_d", "d*x3", "m", "L1_m", "d*m".

model.fit(n_folds=5, max_lags=2, penalty="plugin", method="dml", weighting="identity", seed=None)

ArgumentDefaultMeaning
n_folds5number of folds \(K \ge 3\) (Algorithm 1)
max_lags2residual dates kept in each instrument vector; None = full history
penalty"plugin"first-stage l1 penalty: plug-in rule, "cv", or a number
method"dml""naive" gives the benchmark of Section 5.1
weighting"identity""optimal" = two-step GMM with the J test
seedNoneseed of the random fold split

Results

CallReturns
res.params, res.bse, res.pvalues, res.covcoefficients, standard errors (Eq. A.15), p-values, covariance
res.summary(), .to_latex(), .to_markdown(), .to_frame()the coefficient table in four formats
res.effects(d=1, d_star=0, m=0, period="last")CDE, NDE, NIE, TE and the mediated share
res.derivatives(d=1, m=0, period="last")CDD, NDD, NID, TD and the mediated share
res.bootstrap(n_boot=199, seed=None, n_jobs=1)bootstrap draws; then use se="bootstrap"
eff.wald(rows)joint Wald test of several effects
compare_effects(eff_a, eff_b)paired difference between two specifications
res.plot_coefficients(), res.plot_moments(), res.plot_dose_response(), res.plot_surface(), eff.plot(), eff.plot_profile()figures, each returning (fig, ax)

period accepts "last", a period number, a list, "all" (one row per period) or "average" (Eq. 24).

Application

Complete example with real data

The application of the paper (Section 6): 3,058 U.S. counties, 2001–2019. Treatment: workplace employment rate. Outcome: log personal income per capita. Mediator: log average wage per job. Eight lagged county characteristics are the covariates. All variables are winsorised and standardised, so effects are in standard deviations.

Step 1 — load the data

from panelmed import PanelMediation, compare_effects, load_us_counties
from panelmed.datasets import US_COUNTIES_COVARIATES as X

df = load_us_counties()          # 3,058 counties x 19 years, prepared as in the paper

Step 2 — two specifications

common = dict(data=df, unit="county_id", time="year", basis_degree=3, basis_max_vars=2)

# I: the same effect in every county
spec1 = PanelMediation("y", "m", "d", X, outcome=["d", "m"], mediator=["d"], **common)
res1 = spec1.fit(n_folds=5, max_lags=2, seed=2026)

# II: effects vary with county characteristics
inter = [f"d*{c}" for c in X]
spec2 = PanelMediation("y", "m", "d", X, outcome=["d", "m"] + inter,
                       mediator=["d"] + inter, **common)
res2 = spec2.fit(n_folds=5, max_lags=2, seed=2026)

Specification I uses 99 moment conditions for 3 parameters and Specification II 627 for 19, the same counts as in the paper.

Step 3 — effects averaged over 2001–2019

eff1 = res1.derivatives(d=0, period="average")
eff2 = res2.derivatives(d=0, period="average")
print(eff1.summary())
print(eff2.summary())
Specification I
EstimandCoef.Std.Err.95% CI
NDD (direct)0.2089**0.0875[0.0375, 0.3803]
NID (through wages)0.3997***0.0703[0.2620, 0.5375]
TD (total)0.6086***0.0786[0.4546, 0.7626]
Mediated share
NID / TD0.65680.1199[0.4218, 0.8918]
Specification II
EstimandCoef.Std.Err.95% CI
NDD (direct)0.3657***0.1177[0.1350, 0.5964]
NID (through wages)0.2482***0.0478[0.1545, 0.3419]
TD (total)0.6139***0.1051[0.4080, 0.8198]
Mediated share
NID / TD0.40430.1068[0.1949, 0.6137]

*** p<0.01, ** p<0.05. Analytical standard errors from influence functions (Eq. 21).

Against Table 2 of the paper
NDDNIDTDNID/TD
Specification I, paper0.21450.40170.61620.6519
Specification I, panelmed0.20890.39970.60860.6568
Specification II, paper0.35970.25020.60990.4103
Specification II, panelmed0.36570.24820.61390.4043

The remaining differences come from the random fold split, which the paper does not publish. The paper reports bootstrap standard errors; these are available with res.bootstrap(...).

Step 4 — does the decomposition change?

print(compare_effects(eff1, eff2))
Paired difference, Specification II minus Specification I
EstimandDifferenceStd.Err.P>|z|95% CI
NDD0.1568**0.07620.040[0.0074, 0.3062]
NID-0.1515***0.05100.003[-0.2515, -0.0516]
TD0.00530.06980.940[-0.1315, 0.1421]
NID / TD-0.2525***0.08130.002[-0.4119, -0.0931]

The total effect is the same in both specifications. The share going through wages falls from about two thirds to about 40%.

Step 5 — figures

eff2.plot()
res2.derivatives(d=0, period="all").plot_profile()
res2.plot_dose_response()
res2.plot_surface()
res1.plot_moments()
Dot-and-whisker plot of the direct, indirect and total effects under Specification II
Decomposition under Specification II, 95% intervals.
Year-by-year direct, indirect and total effects with confidence bands, 2001 to 2019
Year-by-year effects: mildly U-shaped, as in Figure A.1 of the paper.
Direct, indirect and total effects as a function of the treatment value
Dose-response of the three effects.
Heatmap of the total effect over pairs of treatment histories
Total effect for every pair \((d, d^{*})\), Parula colours.
Structural coefficients of Specification I with confidence intervals
Structural coefficients, Specification I.
Bar chart of standardised sample moments
Moment diagnostic, Specification I.
Outputs

Interpreting the output

The tables

  • theta[m]: effect of the mediator on the outcome. gamma[d]: effect of the treatment on the mediator. Their product drives the indirect effect.
  • NDE / NDD: effect of the treatment with the mediator kept at the value it would have had without the change.
  • NIE / NID: effect that comes only from the mediator responding to the treatment.
  • TE / TD: always the sum of the two.
  • Mediated share: indirect over total. No test against zero is reported for it.
  • Stars: p-values below 0.01, 0.05, 0.10.

The figures

  • Profile: with constant coefficients, movement over time reflects changes in the covariates that interact with the treatment.
  • Surface: the white line marks pairs of histories with zero effect.
  • Moment diagnostic: bars are standardised sample moments; many bars far outside ±1.96 point to a misspecified dictionary or invalid instruments.
  • Jacobian condition number (table footer): values above about 1,000 signal weak identification.

Pitfall. A stable total effect does not validate its split. Always report the decomposition under more than one dictionary, and check a second seed.

Extensions

Beyond the paper

These are additions to the original method.

Two-step GMM and J test

weighting="optimal" gives efficient GMM and the test of over-identifying restrictions of Hansen (1982).

Mediated share SE

An analytical standard error for indirect/total by the delta method on the influence functions (follows from Theorem 2).

Paired comparison

compare_effects tests whether two specifications differ, from unit-level influence functions or from paired bootstrap draws.

Identification check

The condition number of the moment Jacobian is reported, and a clear error is raised when the rank condition fails.

Not implemented yet: vector-valued mediators (Appendix A.1 of the paper) and unbalanced panels.

Sources

References

  1. Wu, P. and Xiao, Z. (2026). Causal Mediation Analysis for Panel Data with Double/Debiased Machine Learning. arXiv:2610.21339. doi:10.48550/arXiv.2610.21339
  2. Wu, P. and Xiao, Z. (2026). Double/Debiased Machine Learning for Treatment Effects in Dynamic Panels. Journal of the American Statistical Association. doi:10.1080/01621459.2026.2696607
  3. Chernozhukov, V., Chetverikov, D., Demirer, M., Duflo, E., Hansen, C., Newey, W. and Robins, J. (2018). Double/debiased machine learning for treatment and structural parameters. The Econometrics Journal, 21(1), C1–C68. doi:10.1111/ectj.12097
  4. Kapetanios, G. (2008). A bootstrap procedure for panel data sets with many cross-sectional units. The Econometrics Journal, 11(2), 377–395. doi:10.1111/j.1368-423X.2008.00243.x
  5. Hansen, L. P. (1982). Large sample properties of generalized method of moments estimators. Econometrica, 50(4), 1029–1054. doi:10.2307/1912775
  6. Cerqua, A., Letta, M. and Pinto, G. (2026). On the (mis)use of machine learning with panel data. Oxford Bulletin of Economics and Statistics, 88(3), 506–518. doi:10.1111/obes.70019
Author

Author and citation

Cite this package

@software{roudane_panelmed,
  author  = {Roudane, Merwan},
  title   = {{panelmed}: causal mediation analysis
             for panel data with double/debiased
             machine learning},
  year    = {2026},
  url     = {https://github.com/merwanroudane/panelmed},
  version = {0.1.1}
}

Please also cite Wu and Xiao (2026), and Cerqua, Letta and Pinto (2026) when you use the county data.