Learning Under Noise: A Comparison of Three Gradient Estimators in Equilibrium Propagation at Finite Temperature

Description

Learning Under Noise: A Comparison of Three Gradient Estimators in Equilibrium Propagation at Finite Temperature Research Team Khalid Afif Iqnaibi — B.Sc. Computer Science, Third Year Hadi Ayman Shaheen — B.Sc. Computer Engineering, Third Year Amjad Ali Alami — B.Sc. Computer Engineering, Third Year College of Information Technology and Computer Engineering Palestine Polytechnic University Supervisor: Dr. Mousa Farajallah Introduction and Research Problem Equilibrium Propagation (EP) is a learning framework in which a neural network evolves according to an energy function. Learning signals are obtained by comparing the network’s behavior before and after its outputs are weakly pushed toward a desired target. This provides an alternative approach to calculating gradients in energy-based neural networks. At finite temperature, random fluctuations prevent the network from settling into a single fixed state. Learning must therefore rely on statistical averages rather than individual equilibrium states. Three relevant gradient estimators are the one-sided estimator, the symmetric estimator, and the fluctuation-based estimator. Although their mathematical foundations are established, their practical performance depends on temperature, nudging strength, sampling duration, and the time allowed for the system to approach equilibrium. A smaller nudge reduces finite-difference bias but amplifies sampling noise. Longer simulation improves estimation but increases computational cost. In addition, insufficient settling time can produce errors that may be mistaken for effects of temperature. These factors make comparisons difficult unless computational budgets, sampling quality, and reference gradients are carefully controlled. This project will conduct a reproducible comparison of the three estimators under equal simulation budgets. It will investigate when each estimator provides useful learning signals, explain the sources of estimation error, and test whether improved gradient quality translates into better image-classification performance. Literature Review and Research Gap Scellier and Bengio (2017) introduced Equilibrium Propagation and described its stochastic formulation. Their framework connects changes in local energy derivatives under weak nudging to gradients of the expected learning cost. Laborieux et al. (2021) investigated symmetric nudging, which cancels the leading finite-difference bias and improves the small-nudge approximation from first to second order. Massar and Mognetti (2024) developed thermal response identities that express gradients through equilibrium covariances. These identities support a fluctuation-based estimator operating entirely in the free phase. The supplied literature also includes Litman (2025, revised 2026), which interprets finite-nudge updates through a free-energy objective, and De (2026), which reports comparisons involving symmetric EP and wrong-direction one-sided gradients. These recent preprint claims will be treated as findings to reproduce and test. The research gap addressed here is a controlled comparison that evaluates all three estimators against a checked finite-temperature reference, accounts for the cost of every simulation phase, separates systematic bias from random variation, and independently varies temperature and settling time. Objectives 1. Implement and validate one-sided, symmetric, and fluctuation-based EP gradient estimators. 2. Establish analytical and numerical reference gradients with explicit convergence checks. 3. Compare estimator quality across temperatures, nudging strengths, settling lengths, and equal computational budgets. 4. Separate estimation error into systematic bias and random variation. 5. Determine whether wrong-direction gradients arise primarily from insufficient settling, temperature, or their interaction. 6. Test whether gradient-quality differences predict classification performance on MNIST and Fashion-MNIST. 7. Produce reproducible code, configurations, results, and a research manuscript. Mathematical Framework Let x represent the hidden and output states, with the input held fixed, and let w denote the trainable parameters. The network energy is E(x; w), and the supervised cost is: C(x) = ½ ||x_out − y||². Nudging introduces the energy: Fβ(x; w) = E(x; w) + βC(x). At temperature T > 0, overdamped Langevin dynamics are: dx = −∇xFβ(x; w)dt + √(2T)dW. Under suitable confinement and regularity conditions, these dynamics have the stationary Gibbs distribution: pβ(x; w) = exp[−Fβ(x; w)/T] / Zβ(w), where Zβ is the normalizing partition function. The learning objective is the expected free-phase cost: L(w) = ⟨C⟩₀. Define the free energy and average local energy derivative as: A(w, β) = −T log Zβ(w), m(β) = ⟨∇wE⟩β. When the required derivatives and integrals can be interchanged: ∇wA = m(β), ∂A/∂β = ⟨C⟩β. The thermal response identity then gives: g\* = ∇wL(w) \= m′(0) \= −Cov₀(∇wE, C)/T. This identity provides the theoretical connection between nudging-based and fluctuation-based learning. The Three Gradient Estimators One-sided estimator: G1(β) = [m(β) − m(0)] / β. This estimator compares a free phase with a positively nudged phase. Its equilibrium finite-difference bias relative to g\* is O(β). Symmetric estimator: G2(β) = [m(β) − m(−β)] / (2β). This estimator compares positive and negative nudging. Under sufficient smoothness, symmetry cancels the leading error term, giving an equilibrium bias of O(β²). Fluctuation-based estimator: G3 = −Cov₀(∇wE, C)/T. This estimator uses only free-phase fluctuations. At the level of exact equilibrium expectations, it equals g\* and has no nudging bias. Its practical implementation remains affected by finite sampling, autocorrelation, incomplete equilibration, and numerical sampling error. A consistent finite-nudge objective will also be used: Jβ(w) = [A(w, β) − A(w, 0)] / β. Its exact gradient is: ∇wJβ = G1(β). Therefore, G1 will be evaluated against both the expected-cost gradient g\* and the gradient of its own finite-nudge objective. This distinguishes approximation error relative to L from correct optimization of Jβ. For β > 0, the free-energy contrast also satisfies: A(w, β) − A(w, 0) \= β⟨C⟩β + T KL(pβ || p₀). This relationship explains why finite nudging can correspond to a meaningful objective even when its gradient differs from g\*. Research Questions and Hypotheses Q1: How accurately does each estimator recover the reference gradient as temperature, nudge strength, and compute budget change? Q2: How much of the total estimation error comes from systematic bias and how much from random variation? Q3: Do better gradient estimates produce higher classification accuracy under equal simulation budgets? Q4: Are wrong-direction one-sided gradients caused by insufficient settling, thermal noise, or both? The following hypotheses will be preregistered: H1: The symmetric estimator provides better gradient quality than the one-sided estimator under matched computational budgets. H2: The one-sided estimator deteriorates under small nudges, short settling times, and limited sampling budgets. H3: Estimator rankings change with temperature, with the fluctuation-based estimator becoming competitive in some higher-temperature regimes. H4: Better gradient quality predicts better classification performance under matched compute. H5: Insufficient settling contributes more strongly than temperature alone to wrong-direction one-sided gradients. These statements are empirical hypotheses rather than established consequences of the mathematical identities. Methodology Three energy models will be implemented. The first is a solvable quadratic Gaussian model: E(x; w) = ½xᵀA(w)x − b(w)ᵀx, where A is symmetric positive definite. Its equilibrium mean and covariance are: μ = A⁻¹b, Σ = TA⁻¹. If P selects the output states, its expected cost is: L(w) = ½||Pμ − y||² + (T/2)tr(PA⁻¹Pᵀ). This model provides exact expectations and reference gradients without Monte Carlo error. Its gradient may depend on temperature through the covariance contribution. The second model is a layered 784–256–10 network with tanh interactions and a confining quadratic state term. The input is clamped, while hidden and output states evolve dynamically. Interaction terms and weight-sharing conventions will be defined consistently to avoid double-counting connections. The third model adds confining quartic double-well terms to study multimodal landscapes, energy barriers, and slow mixing. The experimental grid will cover temperatures from near zero to 1, five nudging strengths where applicable, five compute budgets, and five settling lengths. At least five independent model seeds will be used, with additional seeds prioritized for headline comparisons when resources permit. Negative nudging will be used only when the resulting energy remains confining. For the specified quadratic-confinement tanh model, symmetric nudging requires β < 1; the common comparison grid will therefore remain below this boundary. The Gaussian model additionally requires A − βPᵀP to remain positive definite. Compute will be counted across every phase, including settling and sampling. Wall-clock runtime will also be recorded because equal simulation-step counts do not necessarily imply identical execution costs. Reference Gradients and Validation Reference gradients will be established through complementary routes: 1. Exact analytical calculations on the Gaussian model. 2. Long Metropolis-adjusted Langevin algorithm (MALA) chains for nonlinear finite-temperature models. 3. Implicit differentiation in the low-temperature, single-minimum regime. For a unique nondegenerate minimum x\* with positive-definite Hessian H, the low-temperature reference is: g\* → −(∇xC)ᵀH⁻¹(∂²E/∂x∂w). This limit will be used only where its assumptions hold. Multimodal models require separate treatment. MALA preserves the intended equilibrium distribution under standard conditions, but finite runs still require mixing and uncertainty checks. Reference runs will use multiple chains, an R-hat threshold below 1.01, effective-sample-size requirements, and agreement checks where alternative references are available. Passing diagnostics will be treated as evidence of convergence rather than proof. Eight automated validation checks will cover analytical gradient agreement, expected bias orders, low-temperature behavior, harmonic-sampler calibration, MALA agreement, gradient signs, convergence reporting, and compute accounting. Euler–Maruyama time-step effects will be tested explicitly. For a harmonic energy E(x) = ½λx², its stationary variance is: Var(x) = T / [λ(1 − λΔt/2)], rather than the continuous-time value T/λ. This provides a direct calibration test and demonstrates why discretization error must be separated from estimator error. Error Analysis and Statistical Evaluation Primary gradient metrics will include cosine similarity, relative error, and the frequency of negative alignment with the reference. Negative alignment indicates an uphill direction to first order for sufficiently small updates. Repeated estimates will support the decomposition: E||ĝ − g\*||² \= ||Eĝ − g\*||² + E||ĝ − Eĝ||². Autocorrelation will be accounted for through effective sample size. For independent phase estimates, finite-difference variance contains a 1/β² amplification factor, explaining why very small nudges can become unreliable. Within the small-nudge regime, the expected error forms are: One-sided: MSE ≈ aβ² + V/β². Symmetric: MSE ≈ bβ⁴ + V′/β². These suggest an intermediate nudge that balances bias and noise. The location of this optimum will be measured rather than assumed. Predictions about temperature-dependent estimator crossovers will be treated as exploratory, particularly outside near-Gaussian regimes. Confidence intervals will resample independent model seeds as clusters, preserving dependence among measurements from the same seed. Small-seed uncertainty will be reported explicitly. Holm correction will be applied to the family of hypothesis-test p-values; effect sizes will be reported separately. Reference runs failing convergence requirements will be logged as unresolved or missing, with their frequency and location reported. They will not be silently discarded. Student Responsibilities and Training Amjad Ali Alami and Hadi Ayman Shaheen will implement the energy models, simulation loops, parallel runs, compute tracking, validation tests, and reference-gradient procedures. They will also lead reproduction experiments, the pilot, and the main parameter sweeps. Khalid Afif Iqnaibi will implement the three estimators, conduct MNIST and Fashion-MNIST training, tune settings using validation data, and lead bias–variance analysis and hypothesis testing. All students will contribute to configuration files, preregistration, documentation, interpretation, and manuscript preparation. Training will cover neural networks, backpropagation, EP, Langevin dynamics, statistical mechanics, MCMC diagnostics, JAX, GPU programming, reproducible configuration management, and experimental statistics. Work Plan and Feasibility Week 1: Review the literature, verify source claims, finalize mathematical definitions, and preregister the experiments. Weeks 2–3: Implement the simulator and estimators and pass all eight validation checks. Week 4: Reproduce selected published or preprint settings and run a pilot to estimate runtime, reference convergence, and resource requirements. Weeks 5–7: Run the main estimator comparisons and independently vary temperature and settling time. Weeks 8–9: Conduct classification experiments and targeted ablations. Week 10: Complete statistical analysis, documentation, and manuscript preparation. The project requires software simulation rather than specialized experimental hardware. Development and pilot experiments will use personal computers. GPU resources for the larger sweeps will be finalized with the supervisor after pilot benchmarking. A reduced initial study using the Gaussian model and the main network at three temperatures will establish whether the full experiment is computationally practical. If resources are limited, grid density will be reduced while preserving independent replication and reference-quality requirements. Expected Contribution and Deliverables The contribution is a controlled empirical evaluation of established EP gradient identities and estimators under finite-temperature simulation. The project will provide: 1. A validated implementation of three estimators and three energy models. 2. Analytical and numerical reference-gradient procedures with documented uncertainty. 3. Equal-budget comparisons showing how temperature, nudging, settling, and sampling affect gradient quality. 4. A separation of systematic bias, sampling variation, and numerical simulation effects. 5. An independent assessment of selected recent claims about finite nudging and wrong-direction gradients. 6. Classification experiments testing the practical relevance of gradient-quality differences. 7. Reproducible configurations, analysis scripts, result summaries, and a research manuscript. Scope and Limitations This is a software study of relatively small energy-based networks. It will not measure physical hardware energy consumption or establish performance for large modern neural architectures. Numerical thermal noise models stochastic dynamics but does not reproduce every source of noise in physical devices. The mathematical identities rely on confinement and regularity assumptions. Numerical references remain subject to finite sampling, and multimodal landscapes may prevent reliable mixing within the available budget. These limitations will be reported alongside the results. Selected References Scellier and Bengio (2017). Equilibrium Propagation. arXiv:1602.05179. Laborieux et al. (2021). Scaling Equilibrium Propagation to deep ConvNets. arXiv:2006.03824. Massar and Mognetti (2024). Equilibrium Propagation: the quantum and the thermal cases. arXiv:2405.08467. Litman (2025; revised 2026). Finite-nudge Equilibrium Propagation in thermal ensembles. arXiv:2511.22024. De (2026). Symmetric Equilibrium Propagation for thermodynamic diffusion training. arXiv:2604.23806. Preprint. Roberts and Tweedie (1996). Supporting work on Langevin sampling and convergence. Gelman and Rubin (1992); Vehtari et al. (2021). Supporting work on MCMC convergence diagnostics. Holm (1979). Supporting work on multiple-hypothesis testing. Efron (1979). Supporting work on bootstrap inference.