0

Is there a standard way to do data analysis? So for example, something like this:

 1. Data mining
 2. Data cleaning
 3. xx
 4. Data and result interpretations

I ask this because I'd like to do it for a paper which analyses a model of Data Science and, therefore, a common method would be appreciated. For a Literature Review, its f.e. (Transfield et. al 2003).

I actually could not find any reliable source, neither an approach to do it standardized.

Shayan Shafiq
  • 1,012
  • 4
  • 11
  • 24
grumpyp
  • 157
  • 5
  • 1
    Hi new member, you are getting downvoted as this is a very open ended question and does not show enough research by yourself. Take a look [at how to ask questions](https://stackoverflow.com/help/how-to-ask) – micstr Dec 30 '20 at 09:24
  • @micstr I did research and added it to the question. In my opinion its not an open question. – grumpyp Dec 30 '20 at 09:42
  • 1
    You might want to google CRISP-DM that is a general framework to work with machine learning projects. – Multivac Dec 30 '20 at 23:27
  • Hey @JulioJesus I guess that's some kind of, what I was looking for. I do some more research in 2021. Thanks so far! – grumpyp Dec 31 '20 at 15:34
  • Might be helpful: https://datascience.stackexchange.com/questions/71816/please-review-my-sketch-of-the-machine-learning-process – Jonathan Jun 15 '21 at 08:16

1 Answers1

0

If you only consider the process of making something from your data (excluding data extraction etc), I'd say something like :

  • Visualising your data to understand it
  • Cleaning your data, removing useless features etc
  • Feature engineering : keeping only relevant features, creating new features from existing ones (combining a few one, making ratio of 2 features for example)
  • Preprocesses (including simplifying your data if it's too precise, like too many classes in a category variable; encoding your variables so it can be taken by a model)
  • Testing and chosing the most adapted model
  • Tuning your model (using cross validation to have the best view of what your model does)
  • Try your model in brand new data (validation set) so you have the best estimation of how good it works, gather your results

This is pretty hard to give 'precise steps' since it differs from one project to another, depending on constraints, type of data, ...

Adept
  • 854
  • 5
  • 17
  • Hi, that sounds like what I was looking for - kind of. But is there something I could quote during this process? So that I say I made this according to someone? – grumpyp Dec 31 '20 at 09:15