Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [methodology]

47 questions
9
votes
2 answers

Ethical consequences of non-deterministic learning processes?

Most advanced supervised learning techniques are non-deterministic by construction. The final output of the model usually depends on some random parts of the learning process. (Random weight initialization for Neural Networks or variable selection /…
Lucas Morin
  • 2,513
  • 5
  • 19
  • 39
7
votes
2 answers

Machine Learning in Practice

I worked on a machine learning project where we dealt with relatively small data sets. I noticed that the way that we tried to increase performance was basically to try out a bunch of different models with different hyperparameters, try out a bunch…
Spencer Gibson
  • 79
  • 1
7
votes
1 answer

How can I learn and apply the scientific method in machine learning?

Rigor Theory. I wish to learn the scientific method and how to apply it in machine learning. Specifically, how to verify that a model captured the pattern in data; how to rigorously reach conclusions based on well-justified empirical…
Mostafa Touny
  • 73
  • 4
4
votes
3 answers

Is removing poorly predicted data points a valid approach?

I'm getting my feet wet with data science and machine learning. Please bear with me as I try to explain my problem. I haven't been able to find anything about this method, but I suspect there's a simple name for what I'm trying to do. To give an…
mtosic
  • 41
  • 2
4
votes
2 answers

Is reseating passengers a reinforcement learning problem?

Requirement is to optimally move passengers from one seat map to another which has a different configuration. Move should be based on many rules like - 1) Families should be sitting together 2) Those who were at windows seat should be moved to…
Aljo Jose
  • 171
  • 4
3
votes
2 answers

How do I learn experimental methodology? When is it relevant?

I just graduated in Computer Science, with a very theoretical background but without any kind of Data Science or Artificial Intelligence experience, and I working on my own to discover those two fields. More precisely, I try to work on a toy…
NowhereToHide
  • 131
  • 1
3
votes
2 answers

Predictive modeling when output affects future input

Assume I have a model which predicts the outcome of the number of icecreams sold in a store. The model is trained on data for the last 5 years while keeping the last year as a validation set and has produced very good results. We now put the model…
CutePoison
  • 450
  • 2
  • 8
3
votes
2 answers

Can you provide examples of business application of vector autoregressive model?

Vector Autoregressive models are exploited at Economics faculties all around the world. They are just another statistical model that solves problem of forecasting, although in a deeply complexity-uncovering manner. Yet to my surprise, there is no…
user2530062
  • 297
  • 2
  • 8
3
votes
0 answers

Ethics of using Existing Algorithms in Competitions

Lets say a paper is published which describes a data science algorithm, and the paper is made available on arxiv (no patent or anything else mentioned in paper). The paper is by a university researcher, and the algorithm is not something well…
Pavel Savine
  • 571
  • 5
  • 13
2
votes
1 answer

Popular classification algorithms over time

In the book "Deep Learning with Python" by Francois Chollet (2018), in section 1.2.4 one can find: Decisions trees learned from data began to receive significant research interest in the 2000s, and by 2010 they were often preferred to kernel…
Ruben Kazumov
  • 211
  • 1
  • 4
2
votes
2 answers

Time-series decomposition to a base level and an effect of another feature

I've got a time-series data (let's denote it as y) and some feature (let's denote it as x). y is dependent on x, but x is often equal to 0. Even then, y is not 0, so we can assume that there's a base level in y which is independent of x.…
jakes
  • 95
  • 12
2
votes
2 answers

Are all problems solvable using machine learning?

I am confronted with a relatively original problem which consists in predicting on which floor of a building audio recordings have been made. I have tried many machine learning approaches but none of them seem to give significant and stable results…
nprime496
  • 65
  • 5
2
votes
1 answer

Can I compare two models trained on different but similar datasets to help find differences between the two datasets?

I have a multivariate dataset the contains A and B. I want to see if there are differences between the A and B samples. I currently have two ideas on how to do this, but I am not sure if they are valid. Train a model on A's samples and separately…
Mike
  • 21
  • 2
1
vote
2 answers

What is the opposite of baseline?

I have created a prediction model and on the one hand I have to compare it with other baseline models, and on the other hand, I have to compare it with the ideal approach (supported by additional data), so I would like to know how I can call it…
joe_mind
  • 25
  • 4
1
vote
2 answers

Is there a good systematic approach to explore and analyze data (prior to modelling)?

I have found a few examples of kernels on Kaggle people have made where they seem to follow a certain methodology in order to systematically analyze and explore the data, to make sure to find all outliers, missing values etc. They are just practical…
Christoffer
  • 121
  • 5
1
2 3 4