Questions tagged [serialisation]
7 questions
5
votes
2 answers
What is a YAML file and where is it used in a machine learning context?
I am not entirely sure if this is on-topic here, so please let me know if it is not. I keep seeing the idea of YAML files pop up while reading machine learning literature. My question is, what exactly is a YAML file, and how does it relate to…
user67797
4
votes
1 answer
PicklingError: Could not serialize object: TypeError: can't pickle fasttext_pybind.fasttext objects
I built a fasttext classification model in order to do sentiment analysis for facebook comments (using pyspark 2.4.1 on windows). When I use the prediction model function to predict the class of a sentence, the result is a tuple with the form…
DS_Tn
- 71
- 1
- 5
2
votes
1 answer
Ways to share Pytorch model without revealing architecture?
We are trying to give a model to collaborators but would like to protect the IP. What are some ways to encrypt/hide/compile the definition when sharing a trained model?
illan
- 21
- 1
1
vote
3 answers
What is a good way to store processed CSV data to train model in Python?
I have about 100MB of CSV data that is cleaned and used for training in Keras stored as Panda DataFrame. What is a good (simple) way of saving it for fast reads? I don't need to query or load part of it.
Some options appear to be:
HDFS
HDF5
HDFS3…
B Seven
- 292
- 1
- 9
1
vote
4 answers
Are scalers or encoders supposed to be serialized along with trained models?
Consider the very basic example below:
X = data.drop("Price", axis = 1)
y = data["Price"]
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size = 0.2, random_state = 1)
scaler = MinMaxScaler()
model =…
Muhammad Usman
- 27
- 5
1
vote
0 answers
model serialization - what is ".pk" format?
I encountered a piece of pytorch code where the trained model is saved with .pk. I often see pytorch models being saved as .pth or .pt. What is .pk format and how is it different from .pth or .pt?
Btw, the following parameters and weights are saved…
JXuan
- 21
- 1
0
votes
1 answer
Use serialized model without installing dependencies
I prototyped an ML model consisting of preprocessing + multiple stacked regressors. I would like a colleague of mine to develop an API that will query the model. Is there any way to query the model (sklearn pipeline) without having to download all…
Rusoiba
- 819
- 4
- 14