Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [automation]

10 questions
5
votes
1 answer

Data Science Pipelines vs Common CD/CL

What is the advantage of Data Science Specific CI/CD (kubeflow, Algo, TFX, mlflow, sagemaker pipelines) vs the already baked flavors that are more generic: Jenkins, Bamboo, Airflow, Google Cloud Build, ... My guess is the Data Science ones give…
brianray
  • 211
  • 2
  • 4
2
votes
1 answer

Automation of finding a starting point of measurement in a large dataset

I am looking for a way to automatically find a starting point of rising in my signal in Python. The data are collected with the frequency 10k (0.0001 s each) so the differences between each point are very small, lost in the noise. I found this point…
tildekara
  • 43
  • 6
2
votes
1 answer

Tool for test/train automation

I need to test different datasets as well as different algorithm implementations. The current workflow looks like: Perform feature extraction from train set Train classifier on this features Feed this classifier to production code Run production…
noname7619
  • 323
  • 2
  • 9
2
votes
3 answers

Are there any frameworks available that allow for automated large scale supervised machine learning?

The typical steps for solving a machine learning/pattern recognition problem: Data Analysis and splitting the data into test and train sets. Choosing a model. Training the model, and testing the model against the test set. If the accuracy of the…
Alex S Kinman
  • 477
  • 3
  • 12
2
votes
1 answer

Automate naming of scanned documents

(If this is not the right forum for this, I apologize) I'm trying to write a program that can automatically name scanned PDF files depending on what text they contain on their first page. I have ~10,000 PDFs that are already named the way they…
DoubleR
  • 21
  • 2
1
vote
1 answer

Auto-scheduling with Machine Learning

I am very new to Data Science, but I have an use case which I want to solve. I want to build a data synchronization scheduler which keeps track of the amount of data sync after every scheduled triggers and auto-adjusts the next schedule. For…
abhishek
  • 113
  • 4
1
vote
0 answers

Automate detection of overfitting models based on autoML libraries

I'm trying to use machine learning to impute missing data in series using some auto-ML libraries in python (so far : dabl, FLAML, auto-sklearn and AutoKeras). I know the way to detect overfitting in a "classical" way: that would be to plot…
tgrandje
  • 111
  • 4
0
votes
1 answer

Automated bug finder through machine learning?

Assume that I have a large software application, this application is event driven i.e. when a user configures something by pressing on a button a code flow executes. Along with this application I also have a system which can emulate a user and can…
Gaux
  • 1
  • 2
0
votes
0 answers

Is it possible to use a python script to open and run a Tableau Prep Flow?

My organization utilizes a tableau dashboard embedded in our webpage, which is manually updated on a regular basis. I am working on automating this process, with the hope of having it updated automatically and on a more frequent basis. We have a…
SchrodingersStat
  • 165
  • 5
0
votes
0 answers

Are there tools that would run many machine learning fits without much setup?

Are there tools that would run many machine learning fits without much setup? Such tool could be useful for discovering what kind of model could be the best fit, without trying each of them manually.
mavavilj
  • 416
  • 1
  • 3
  • 12