Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [scipy]

SciPy is a python ecosystem of open-source packages for scientific computation. Some of the core packages include NumPy, SciPy, Matplotlib, IPython, SymPy, and pandas.

SciPy is a python ecosystem of open-source packages for scientific computation. Some of the core packages include NumPy, SciPy, Matplotlib, IPython, SymPy, and pandas.

92 questions
11
votes
2 answers

Understanding scipy sparse matrix types

I am trying to select the best scipy sparse matrix type to use in my algorithm. In that, I should initialize data in a vij way, then I should use it to perform matrix vector multiplication. Eventually I have to add rows and cols. Trying to select…
Federico Caccia
  • 760
  • 6
  • 18
6
votes
3 answers

What is the best algorithm/solution for predicting the following?

I have a dataset that comprises 76 countries, and 6 columns of distinct quantitative variables, which are the mean values of that variable relative to each country: If I were to take a random sample of the 6 variables - an individual within one of…
free_road
  • 71
  • 4
5
votes
1 answer

Influence of trend on (supposedly) correlated time series

TL;DR: What is the impact of a linear trend on the correlation between time series that are (most likely) not spuriously correlated? I'm currently trying to reconstruct/cross-validate an analysis delivered by one of my companies contractors. The…
Viktor Katzy
  • 153
  • 5
4
votes
2 answers

Can I use scipy.optimize module with PyTorch?

I need to use scipy.optimize module after encoding some data with pytorch. However, scipy.optimize does not take torch.tensor as its input. How can I use scipy.optimize module with data with gradient path attached?
Eiffelbear
  • 175
  • 8
4
votes
4 answers

Using Machine Learning to Predict Temperature

I am a beginner in ML and I want to create a smart thermostat, that after collecting enough data from the interaction with the user, it will start to set the home temperature by itself. What I got so far is the hardware prototype that lets the user…
Emanuel Giurgiu
  • 43
  • 1
  • 3
4
votes
1 answer

Compute parameters of a PDF (probability density function) for which no closed form expression is available

I would like to compute parameters such as mean, variance, quantiles, etc. for a PDF which is only given as a piece of code. That is, it can only be evaluated numerically at given points; no closed-form expression. For example, after using…
Konstantin
  • 163
  • 8
3
votes
0 answers

Scipy minimization failing with inequality constraints or bounds

I am trying to use scipy.optimize to solve a minimization problem but getting failures on using an inequality constraint or a bound. Looking for any suggestions regarding proper usage of constraints vs bounds, and if any other algorithm would be…
dayum
  • 131
  • 4
3
votes
1 answer

Smaller alternatives to sklearn that doesn't require scipy?

I am packaging my model for deployment in aws lambda which has a size limit of 250mb for all dependencies. Sklearn, if you include its dependencies of numpy and scipy is a huge package. Are there any alternatives to sklearn that don't require scipy…
coderboi
  • 33
  • 3
3
votes
0 answers

Converting pandas dataframe to scipy sparse arrays

Converting pandas data frame with mixed column types -- numerical, ordinal as well as categorical -- to Scipy sparse arrays is a central problem in machine learning. Now, if my pandas' data frame consists of only numerical data, then I can simply do…
learner
  • 359
  • 1
  • 11
3
votes
2 answers

How to interpret ANOVA results?

I am trying to identify what attributes are not relevant in my dataset to remove them before fitting a classifier. The target is a categorical variable with three different values. I also have a lot of numerical attributes. For ANOVA, I used the…
Tlaloc-ES
  • 337
  • 1
  • 6
3
votes
1 answer

Incremental clustering algorithm

I am looking for an incremental clustering algorithm. By incremental I mean an algorithm that builds clusters starting from an initial dataset and that is able to progressively ingest new items/observations adding them to existing or new…
Sirion
  • 131
  • 2
3
votes
1 answer

Detect max extreme peaks/valleys with min 5% vertical delta

I have been using scipy.signal.argrelextrema but I'm not getting the desired results. My goal is to: identify the peaks/valleys in a 1D timeseries that are a minimum of 5% away from the prior peak/valley. The following snippet computes the extreme…
Mark
  • 53
  • 4
2
votes
0 answers

What would be a good randomization environment for data science?

I would like to know if there are any best practices to optimize random environment. Currently I use this simple structure in my config : from numpy.random import Generator, PCG64 rng = Generator(PCG64(42)) np.random.seed(42) I use the rng…
Al_P
  • 21
  • 1
2
votes
1 answer

Dendrogram: ValueError: The truth value of a Series is ambiguous. Use a.empty, a.bool(), a.item(), a.any() or a.all()

I am trying to plot a Dendrogram to cluster data but this error is stopping me. My datea is here. I first chose columns to work with: df_euro =…
Sam.H
  • 21
  • 2
2
votes
1 answer

Which is the best algorithm for entity extraction for unstructured document

I have unstructured documents from which I have to extract the information like let buyer name, seller name, expiry date, buying date etc. I had planned to use spacy(Custom entity recolonization(Followed this blog…
Rajesh das
  • 113
  • 7
1
2 3 4 5 6 7