Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [hyperparameter-tuning]

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm.

Hyperparameter tuning (also called hyperparameter optimization) refers to the process of finding the optimal set of hyperparameters for a given machine learning algorithm. Popular methods for searching for the optimal hyperparameters including: Grid Search, Random Search, Bayesian Optimization, Gradient-Based Optimization, and Early Stopping. Finding the optimal set of hyperparameters can often significantly improve a model's performance.

283 questions
40
votes
6 answers

How to set the number of neurons and layers in neural networks

I am a beginner to neural networks and have had trouble grasping two concepts: How does one decide the number of middle layers a given neural network have? 1 vs. 10 or whatever. How does one decide the number of neurons in each middle layer? Is it…
stk1234
  • 573
  • 1
  • 5
  • 6
21
votes
2 answers

How to adjust the hyperparameters of MLP classifier to get more perfect performance

I am just getting touch with Multi-layer Perceptron. And, I got this accuracy when classifying the DEAP data with MLP. However, I have no idea how to adjust the hyperparameters for improving the result. Here is the detail of my code and…
Irving.ren
  • 327
  • 1
  • 2
  • 7
16
votes
3 answers

Why my network needs so many epochs to learn?

I'm working on a relation classification task for natural language processing and I have some questions about the learning process. I implemented a convolutional neural network using PyTorch, and I'm trying to select the best hyper-parameters. The…
user3319400
  • 261
  • 1
  • 2
  • 6
13
votes
2 answers

What makes a Tree-Structured Parzen Estimator "tree-structured?"

From what I understand the Tree-Structured Parzen Estimator (TPE) creates two probability models based on hyperparameters that exceed the performance of some threshold and hyperparameters that don't. What I don't fully understand is why TPE is…
AAC
  • 499
  • 2
  • 5
  • 13
11
votes
2 answers

What is the most efficient method for hyperparameter optimization in scikit-learn?

An overview of the hyperparameter optimization process in scikit-learn is here. Exhaustive grid search will find the optimal set of hyperparameters for a model. The downside is that exhaustive grid search is slow. Random search is faster than grid…
Brian Spiering
  • 20,142
  • 2
  • 25
  • 102
10
votes
4 answers

Which is first ? Tuning the parameters or selecting the model

I've been reading about how we split our data into 3 parts; generally, we use the validation set to help us tune the parameters and the test set to have an unbiased estimate on how well does our model perform and thus we can compare models based on…
Ahmed Gharbi
  • 103
  • 1
  • 6
8
votes
3 answers

How to combine GridSearchCV with Early Stopping?

I'm a beginner in machine learning and want to train a CNN (for image recognition) with optimized hyperparameter like dropout rate, learning rate and number of epochs. The optimal hyperparameter I try to find via GridSearchCV from Scikit-learn. I…
Code Now
  • 393
  • 5
  • 10
7
votes
1 answer

how to pass parameters over sklearn pipeline's stages?

I'm working on a deep neural model for text classification using Keras. To fine tune some hyperparameters i'm using Keras Wrappers for the Scikit-Learn API. So I builded a Sklearn Pipeline for that: def create_model(optimizer="adam",…
Amine Benatmane
  • 241
  • 2
  • 9
7
votes
1 answer

How to decide how many n_neighbors to consider while implementing LocalOutlierFactor?

I have a data set with rows: 134000 and columns: 200. I am trying to identify the outliers in data set using LocalOutlierFactor from scikit-learn. Although I understand how the algorithm works, I am unable to decide n_neighbors for my data…
Neha Bhushan
  • 121
  • 1
  • 3
6
votes
1 answer

Why do BERT classification do worse with longer sequence length?

I've been experimenting using transformer networks like BERT for some simple classification tasks. My tasks are binary assignment, the datasets are relatively balanced, and the corpus are abstracts from PUBMED. The median number of tokens from…
Hooked
  • 207
  • 1
  • 6
6
votes
3 answers

Scaling neural networks

While using Neural Networks (TensorFlow: Deep Neural Regressor), when increasing your training data size from a sample to the whole data (say a 10x larger dataset), what changes should you make to the model architecture (deeper/wider), learning rate…
Sharan
  • 61
  • 3
6
votes
3 answers

Which parameters are hyper parameters in a linear regression?

Can the number of features used in a linear regression be regarded as a hyperparameter? Perhaps the choice of features?
Vykta Wakandigara
  • 213
  • 1
  • 2
  • 6
5
votes
1 answer

How to make it possible for a neural network to tune its own hyper parameters?

I am curious about what would happen to hyperparameters when they would be set by a neural network itself or by creating a neural network that encapsulates and influences the hyperparameters of the network it encapsulates. The goal for me here is to…
user3473161
  • 53
  • 3
5
votes
2 answers

How to choose the random seed?

I understand this question can be strange, but how do I pick the final random_seed for my classifier? Below is an example code. It uses the SGDClassifier from SKlearn on the iris dataset, and GridSearchCV to find the best random_state: from…
Bruno Lubascher
  • 3,488
  • 1
  • 11
  • 35
5
votes
4 answers

Hyperparameter tuning in multiclass classification problem: which scoring metric?

I'm working with an imbalanced multi-class dataset. I try to tune the parameters of a DecisionTreeClassifier, RandomForestClassifier and a GradientBoostingClassifier using a randomized search and a bayesian search. For now, I used just accuracy for…
Christian
  • 151
  • 3
1
2 3
…
18 19