Bayesian statistics is a statistical paradigm that contrasts with that of frequentist statistics. Bayesian methods rely on prior information do determine the degree of belief in the probability of a value.
Questions tagged [bayesian]
143 questions
13
votes
2 answers
What makes a Tree-Structured Parzen Estimator "tree-structured?"
From what I understand the Tree-Structured Parzen Estimator (TPE) creates two probability models based on hyperparameters that exceed the performance of some threshold and hyperparameters that don't.
What I don't fully understand is why TPE is…
AAC
- 499
- 2
- 5
- 13
6
votes
1 answer
Changing the batch size during training
The choice of batch size is in some sense the measure of stochasticity :
On one hand, smaller batch sizes make the gradient descent more stochastic, the SGD can deviate significantly from the exact GD on the whole data, but allows for more…
spiridon_the_sun_rotator
- 294
- 1
- 7
6
votes
1 answer
Modeling uncertainty from Logistic Regression
Logistic regression is a part in a simulation pipeline that I use for some scenario analysis. The dataset that this is based on is not small but relatively noisy, and only one explanatory variable/feature. Of course I can say something about this…
Jan van der Vegt
- 9,328
- 34
- 52
6
votes
3 answers
Which tribe does Probabilistic Graphical Models fall under?
Pedro Domingos in "The Master Algorithm" listed five tribes of machine learning algorithms:
Symbolists
Connectionists
Evolutionaries
Bayesians
Analogizers
Which category do probabilistic graphical models fall under?
From wikipedia…
alvas
- 2,340
- 6
- 25
- 38
5
votes
2 answers
How useful is Bayesian Inference
Last few months, I had been exposed to Bayesian Inference in ML course
With further investigation, I come to place where there is MCMC technique to simulate the posterior distribution.
It seems interesting. However, I am not sure if it is really…
chris tan
- 53
- 3
5
votes
1 answer
Does the Bayesian MAP give a probability distribution over unseen data?
I'm working my way through the Bayesian world. So far I've understood that the MLE or the MPA are point estimates, therefore using such models just outputs one specific value and not a distribution.
Moreover, vanilla neural networks do in fact…
Micha Christ
- 51
- 1
5
votes
2 answers
Effect of outliers on Naive Bayes
Are Naive Bayes algorithms affected by outliers in the data? Suppose there is a data set, does one need to remove outliers before applying Naive Bayes?
James Smith
- 51
- 1
- 2
4
votes
2 answers
High Recall but too low Precision result in imbalanced data
I was training a model using XGBoost Classifier on a heavy imbalanced database with 232:1 of binary class. Because my training data contains 750k rows and 320 features (after doing many feature engineering, feature correlation filtering, and low…
zonna
- 73
- 7
4
votes
1 answer
How is bayesian risk computed to prune decision trees?
I've been trying to follow this paper on Bayesian Risk Pruning. I'm not very familiar with this type of pruning, but I'm wondering a few things:
(1) The paper describes risk-rates to be defined per example. We have $R_k(a_i|x)=\sum\limits_{j=1,j…
nrael
- 41
- 7
4
votes
2 answers
Mean estimation for nested location data
I want to estimate the average income for a location. I have nested data in the following way: A block is inside a neighborhood, which is inside a zipcode, which is inside a district, which is inside a region, which is inside a state.
I want to…
David Masip
- 5,981
- 2
- 23
- 61
4
votes
1 answer
Bayesian regularization vs dropout for basic ann
Does it make sense conceptually to apply dropout to an artificial neutral network while also applying bayesian regularization?
On one hand I would think that technically this should work just fine, but on the other hand if bayesian regularization is…
Bram
- 143
- 2
4
votes
3 answers
What is the meaning of likelihood?
I am studying Bayes probability applied to machine learning, and I have encoutered the concept of likelihood, which I don't understand.
I have seen that the Bayes rule is:
$P(A|B)=\frac{P(B|A)P(A)}{P(B)}$
where $P(B|A)P(A)$ is the conditional…
J.D.
- 841
- 4
- 15
- 29
4
votes
2 answers
Bayesian optimisation in deeplearning
Has anyone tried using Bayesian optimisation to get best learning rates, and other hyperparameters for deeplearning.
How to change the parameters between the training. Any examples on callbacks?
Can I see some codes to implement them?
vizakshat
- 465
- 2
- 5
- 16
4
votes
1 answer
How to best estimate the coefficients of a confusion matrix in case of strong class imbalance?
I have a trained binary classifier (forget about how this was trained and think of it as a magical black box) and I would like to measure its classification performance (e.g. compute a confusion matrix) on a strongly imbalanced population/test…
Davide Fiocco
- 421
- 3
- 12
4
votes
1 answer
Why the estimated Lasso coefficients of almost all variables are equal to zero?
I would greatly appreciate if you could guide me. In fact, I used "Bayesian Optimization " to tune hyper-parameters of Lasso but the estimated Lasso coefficients of almost all variables are equal to zero.
X =…
ebrahimi
- 1,277
- 7
- 20
- 39