Questions tagged [entropy]

Can someone practically explain the rationale behind Gini impurity vs Information gain (based on Entropy)? Which metric is better to use in different scenarios while using decision trees?

I have a binary labelled dataset with numeric features. I want to create a "business rule" of the type y = x1 > t1 and x2 > t2 and x3 > t3. X are my features and I just have to find which are the thresholds t that fit the data better. I have tried…

I'm reading a paper about model uncertainty quantification. Specifically, it says epistemic uncertainty is a kind of uncertainty due to lack of knowledge about a particular region in the input space. Also, it makes a mathematical characterization of…

Stable diffusion is essentially removing entropy from noisy images. If you have $n$ latent dimensions, and add Gaussian noise independently to each dimension, then the added entropy is on the order of $n\log\sigma$, where $\sigma$ is the variance of…