Problem for a math formula in Weight Uncertainty in Neural Network

Question

I am studying the paper https://arxiv.org/pdf/1505.05424.pdf and there is a formula I don't get page 4:

I don't understand how they obtain this formula. Moreover, with chain rule, I get $\frac{\partial f(\mathrm w, \theta)}{\partial\mathrm w} = \frac{\partial f(\mathrm w, \theta)}{\partial \mu}$.

Could someone tell me the proof behind the formula? I am not very good with differentiation

Answer 1

import numpy as np from scipy.stats import norm

Generate some random data

np.random.seed(42) data = np.random.normal(0, 1, size=100)

Define prior distribution

prior_mean = 0 prior_std = 1

Define variational posterior distribution

posterior_mean = 0 posterior_std = 1

Perform variational inference

for _ in range(1000): # Number of iterations for optimization # Update mean and standard deviation of the variational posterior posterior_mean = np.mean(data) / (1 + 1 / prior_std2) posterior_std = np.sqrt(1 / (1 / prior_std2 + len(data) / posterior_std**2))

Calculate variational posterior probabilities

x_values = np.linspace(-5, 5, num=100) posterior_probs = norm.pdf(x_values, loc=posterior_mean, scale=posterior_std)

Print mean and standard deviation of variational posterior

print("Variational Posterior Mean:", posterior_mean) print("Variational Posterior Standard Deviation:", posterior_std)