Data Science Stack Exchange
Data Science Stack Exchange

Questions tagged [rnn]

A recurrent neural network (RNN) is a class of artificial neural network where connections between units form a directed cycle.

744 questions
123
votes
2 answers

Training an RNN with examples of different lengths in Keras

I am trying to get started learning about RNNs and I'm using Keras. I understand the basic premise of vanilla RNN and LSTM layers, but I'm having trouble understanding a certain technical point for training. In the keras documentation, it says the…
Tac-Tics
  • 1,350
  • 2
  • 8
  • 6
86
votes
8 answers

Time series prediction using ARIMA vs LSTM

The problem that I am dealing with is predicting time series values. I am looking at one time series at a time and based on for example 15% of the input data, I would like to predict its future values. So far I have come across two models: LSTM…
ahajib
  • 1,075
  • 1
  • 9
  • 15
59
votes
5 answers

Number of parameters in an LSTM model

How many parameters does a single stacked LSTM have? The number of parameters imposes a lower bound on the number of training examples required and also influences the training time. Hence knowing the number of parameters is useful for training…
wabbit
  • 1,297
  • 2
  • 12
  • 15
38
votes
4 answers

What is the meaning of "The number of units in the LSTM cell"?

From Tensorflow code: Tensorflow. RnnCell. num_units: int, The number of units in the LSTM cell. I can't understand what this means. What are the units of LSTM cell? Input, Output and Forget gates? Does this mean "the number of units in the…
Brans Ds
  • 849
  • 1
  • 8
  • 17
36
votes
1 answer

Paper: What's the difference between Layer Normalization, Recurrent Batch Normalization (2016), and Batch Normalized RNN (2015)?

So, recently there's a Layer Normalization paper. There's also an implementation of it on Keras. But I remember there are papers titled Recurrent Batch Normalization (Cooijmans, 2016) and Batch Normalized Recurrent Neural Networks (Laurent, 2015).…
Rizky Luthfianto
  • 2,176
  • 2
  • 19
  • 22
31
votes
6 answers

Validation loss is not decreasing

I am trying to train a LSTM model. Is this model suffering from overfitting? Here is train and validation loss graph:
DukeLover
  • 561
  • 1
  • 6
  • 14
24
votes
2 answers

What's the difference between the cell and hidden state in LSTM?

LSTM cells consist of two types of states, the cell state and hidden state. How do cell and hidden states differ, in terms of their functionality? What information do they carry?
user105907
22
votes
1 answer

Understanding Timestamps and Batchsize of Keras LSTM considering Hiddenstates and TBPTT

What I'm trying to do What I am trying to do is predicting the next data-point $x_t$ for each point in the timeseries $[x_0, x_1, x_2,...,x_T]$ in the context of a date-stream in real-time, in theory the series is infinity. If a new value $x$ is…
KenMarsu
21
votes
3 answers

What is LSTM, BiLSTM and when to use them?

I am very new to Deep learning and I am particularly interested in knowing what are LSTM and BiLSTM and when to use them (major application areas). Why are LSTM and BILSTM more popular than RNN? Can we use these deep learning architectures in…
Volka
  • 711
  • 3
  • 6
  • 21
19
votes
4 answers

What is the difference between word-based and char-based text generation RNNs?

While reading about text generation with Recurrent Neural Networks I noticed that some examples were implemented to generate text word by word and others character by character without actually stating why. So, what is the difference between RNN…
minerals
  • 2,137
  • 3
  • 17
  • 19
17
votes
3 answers

How to determine feature importance in a neural network?

I have a neural network to solve a time series forecasting problem. It is a sequence-to-sequence neural network and currently it is trained on samples each with ten features. The performance of the model is average and I would like to investigate…
Aesir
  • 438
  • 1
  • 6
  • 14
16
votes
1 answer

Multi-dimentional and multivariate Time-Series forecast (RNN/LSTM) Keras

I have been trying to understand how to represent and shape data to make a multidimentional and multivariate time series forecast using Keras (or TensorFlow) but I am still very unclear after reading many blog posts/tutorials/documentation about how…
Bastien
  • 263
  • 1
  • 2
  • 6
15
votes
2 answers

Dropout on which layers of LSTM?

Using a multi-layer LSTM with dropout, is it advisable to put dropout on all hidden layers as well as the output Dense layers? In Hinton's paper (which proposed Dropout) he only put Dropout on the Dense layers, but that was because the hidden inner…
BigBadMe
  • 740
  • 1
  • 6
  • 18
15
votes
1 answer

Why do we need to add START + END symbols when using Recurrent Neural Nets for Sequence-to-Sequence Models?

In the Sequence-to-Sequence models, we often see that the START (e.g. ) and END (e.g. ) symbols are added to the inputs and outputs before training the model and before inference/decoding unseen data. E.g.…
alvas
  • 2,340
  • 6
  • 25
  • 38
15
votes
1 answer

RNN using multiple time series

I am trying to create a neural network using time series as input, in order to train it based on the type of each series. I read that using RNNs you can split the input into batches and use every point of the time series into individual neurons and…
Ploo
  • 323
  • 2
  • 8
1
2 3
…
49 50