Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [cloud-computing]

26 questions
40
votes
6 answers

Are there free cloud services to train machine learning models?

I want to train a deep model with a large amount of training data, but my desktop does not have that power to train such a deep model with these abundant data. I'd like to know whether there are any free cloud services that can be used for training…
Green Falcon
  • 13,868
  • 9
  • 55
  • 98
34
votes
5 answers

What are the use cases for Apache Spark vs Hadoop

With Hadoop 2.0 and YARN Hadoop is supposedly no longer tied only map-reduce solutions. With that advancement, what are the use cases for Apache Spark vs Hadoop considering both sit atop of HDFS? I've read through the introduction documentation for…
idclark
  • 521
  • 1
  • 5
  • 6
8
votes
1 answer

How to download dynamic files created during work on Google Colab?

I have two different files and on the first, I tried to save data to file as: np.save(open(Q1_TRAINING_DATA_FILE, 'wb'), q1_data) On second file, i'm trying to load it the same way using: q1_data = np.load(open(Q1_TRAINING_DATA_FILE, 'rb')) I then…
vikbehal
  • 185
  • 1
  • 1
  • 7
3
votes
2 answers

Alternatives to GCP / AWS / Azure

Can anyone recommend an alternative to the big 3 cloud computing alternatives? I know they're the best but I found them overly complicated because they cater to massive enterprises. The amount of set up required just to get an instance running is…
lara_toff
  • 211
  • 1
  • 2
3
votes
2 answers

Different number of images in classes

I am working on a deep learning CNN project. The dataset contains more than 500 classes and the classes have different numbers of items (images). For example, some of the classes have 5 images and some of the classes have 10 and some of the classes…
N.IT
  • 1,975
  • 4
  • 17
  • 35
2
votes
1 answer

Train a deep reinforcement learning model using two computers

I would like to know if there is a way to train a deep rl model using two different computers. The first one would execute the game and send requisitions to the second computer which would store and train the model itself. Obs: The computers aren't…
Matheus Prandini
  • 107
  • 6
2
votes
0 answers

How much data and computation power do I need to train a machine translation model using Transformer architecture?

I am working right now on creating a dataset to use in creating a machine translation model to translate between two dialects. I have two questions that I am trying to find an answer for: How much data is enough? how many pairs of sentences would…
user13288833
  • 21
  • 1
1
vote
2 answers

Deep learning on cloud

I am trying to implement some deep learning models with large amount of data around 10gigabyte. Although, my Laptop and Collab-free crashes when it tries to load them. Do you think it worths to buy collab-pro? Do you suggest any other solutions? But…
mlds1337
  • 11
  • 2
1
vote
1 answer

Keras multi GPU in vast.ai

I am trying to run a keras model on vast.ai using multiple GPUs. For that I am using keras.utils.multi_gpu_model , however I keep having this error: if multi_GPU and n_GPUs > 1: model = multi_gpu_model(model) AttributeError: module…
Joaquin Rives
  • 11
  • 1
1
vote
0 answers

What VM should I use?

So I am starting a project that has some code in it that has to do with machine learning (for example recommender system). At the moment I am thinking that if I want to deploy this application, should I look into the VM´s that are dedicated to…
v3tra
  • 11
  • 1
1
vote
0 answers

Moving paper documents to cloud (AWS) -

I'm trying to solve a simple problem and seeking advise/suggestions - I have lot of paper documents in my drawers which I want to store on cloud (S3) as PDFs. My idea blueprint is - 1) scanning all of them using printer 2) store it locally in…
ranit.b
  • 451
  • 4
  • 15
1
vote
0 answers

What is the easiest way to duplicate my model and run in 10 cloud machines?

I have built a model in a cloud machine(google cloud). It runs for a few hours to a day. I need to scan some parameters like learning rate and batch size. How do I duplicate my compute engine, run the model with different parameters, collect the…
user3226167
  • 111
  • 3
1
vote
0 answers

How to Read Photos for Deep Learning on Google Cloud Platform Compute Engine VM Instance, Through Jupyter Lab

I've set up a project on GCP with a Compute Engine VM and Storage Bucket. Access Scopes set to Allow full access to all Cloud APIs Have set a default Region and Zone I believe I have completed SSH authorization but not 100% sure given the…
Matan
  • 81
  • 1
  • 4
1
vote
1 answer

how to run my python code on google cloud without fear of getting disconnected - an absolute beginner?

I have been trying to use python 3 for text mining on a 650 MB csv file, which my computer was not powerful enough to do. My second solution was to reach out to google cloud. I have set up my VMs and my jupyter notebook on google cloud, and it works…
Pie-ton
  • 113
  • 5
1
vote
0 answers

Is there a way to use Orange on a remote server or in the cloud?

My question there is a way to run the Orange, not on a local computer, but on a remote server, even in the cloud? My notebook has only 8GB memory and limited number of CPU core, which is insufficient for real-world data mining task, however I can…
obalasz
  • 11
  • 1
1
2