Data Science Stack Exchange
Data Science Stack Exchange

Most Popular

more tags
1500 questions
8
votes
3 answers

Why do I get an OOM error although my model is not that large?

I am a newbie in GPU based training and deep learning models. I am running cDCGAN (Conditional DCGAN) in TensorFlow on my 2 Nvidia GTX 1080 GPUs. My data set consists of around 320,000 images with size 64*64 and 2,350 class labels. If I set my batch…
Ammar Ul Hassan
  • 185
  • 1
  • 1
  • 5
8
votes
5 answers

When to use mean vs median

I'm new to data science and stats, so this might seems like a beginner question. I'm working on a dataset where I've user's Twitter followers gain per day. I want to measure the average growth he had over a period of time, which I did by finding the…
Mukul Jain
  • 193
  • 1
  • 6
8
votes
2 answers

dataframe.columns.difference() use

I am trying to find the working of dataframe.columns.difference() but couldn't find a satisfactory explanation about it. Can anyone explain the working of this method in detail?
Parth S.
  • 83
  • 1
  • 1
  • 5
8
votes
1 answer

Micro-F1 and Macro-F1 are equal in binary classification and I don't know why

I have a binary classification problem which in the test set, the number of data in both classes are equal (the test number of class 0 and class 1 are equal). Since we know that the number of samples from every class are equal, I use median on the…
user137927
  • 379
  • 1
  • 3
  • 10
8
votes
2 answers

Date Extraction in Python

I would like to extract all date information from a given document. Essentially, I guess this can be done with a lot of regexes: 2019-02-20 20.02.2019 ("German format") 02/2019 ("February 2019") "tomorrow" (datetime.timedelta(days=1)) "yesterday"…
Martin Thoma
  • 18,630
  • 31
  • 92
  • 167
8
votes
1 answer

What is GridSearchCV doing after it finishes evaluating the performance of parameter combinations that takes so long?

I'm running GridSearchCV to tune some parameters. For example: params = { 'max_depth':[18,21] } gscv = GridSearchCV( xgbc, params, scoring='roc_auc', verbose=50, cv=StratifiedKFold(n_splits=2,…
Dan Scally
  • 1,724
  • 6
  • 23
8
votes
3 answers

Is a "curve" considered "linear"?

In linear regression, we are fitting a polynomial to a set of data points. In Bishop's book of Pattern Recognition & Machine Learning, there are a few examples where the fit is a curve or a straight line. I am a bit confused if a curve is linear or…
Srishti M
  • 469
  • 4
  • 9
8
votes
4 answers

How to handle features which are not always available?

I have a feature in my feature vector that is not always available respectively sometimes (for some samples) it makes no sense to use it. I feed a sklearn MLPClassifier with this feature vector. Does the neural network learn by itself when the…
jochen6677
  • 561
  • 2
  • 4
  • 9
8
votes
3 answers

Are there any actual, commercial uses of GANs already?

Doing research on the internet, I found many scientific papers, ideas, and experiments concerning GANs. But I was unable to find a single example of it being already used commercially. Q1 can you give examples of companies already using GANs in…
Przemysław Czechowski
  • 181
  • 4
8
votes
1 answer

How does dropout work during testing in neural network?

The below paragraph is picked from the textbook Hands-On Machine Learning with sci-kit learn & Tensorflow. I couldn't understand what the author is trying to convey. It would be really grateful if someone can provide an explanation for below…
James K J
  • 447
  • 1
  • 5
  • 15
8
votes
1 answer

How to handle preprocessing (StandardScaler, LabelEncoder) when using data generator to train?

So, I have a dataset that is too big to load into memory all at once. Therefore I want to use a generator to load batches of data to train on. In this scenario, how do I go about performing scaling of the features using LabelEncoder +…
Jim
  • 181
  • 3
8
votes
0 answers

Training value neural network AlphaGo style

I have been trying to replicate the results obtained by AlphaGo following their supervise learning protocol. The papers specify that they use a network that has two heads: a value head that predicts the winner of the game and a policy head that…
Phaune
  • 101
  • 4
8
votes
1 answer

How to implement a Fourier Convolution layer in keras?

I'm currently investigating the paper FCNN: Fourier Convolutional Neural Networks. The main contribution of the paper is that CNN training is entirely shifted to the Fourier domain without loss of effectiveness. The proposed architecture looks as…
deepsnow
  • 83
  • 1
  • 3
8
votes
2 answers

Obtaining a confidence interval for the prediction of a linear regression

The data I am working with is being used to predict the duration of a trip between two points. There are about 100 different trips in the data and ~90k observations. I am using the standard pattern: feature_cols = df_features.columns.drop( […
ericg
  • 181
  • 1
  • 1
  • 2
8
votes
5 answers

In a binary classification, should the test dataset be balanced?

I have a dataset with 4519 samples labeled as "1", and 18921 samples labeled as "0" in a binary classification exercise. I am well aware that during the training phase of a classification algorithm (in this case, a Random Forest) the number of 0/1…
Irene
  • 193
  • 1
  • 5
1 2 3
…
99 100