1

I see this code concept(with Keras library) in most code examples of LSTM:

model.add(LSTM(X))
model.add(Dense(Y))

But I don't really know if I have 10 time-steps in input side and need the last time-step in output is the following code true?

model.add(LSTM(10))
model.add(Dense(1))

Or if I have 10 input time-steps and need 10 output time-steps(I mean I need all outputs from all units) is the following code true?

model.add(LSTM(10))
model.add(Dense(10))

How should I specify if I have 1 layer LSTM with 10 units, I need all of 10 outputs? Or I need 1 last output from unit 10? Or the last 5 outputs from unit 5 to 10?

user3486308
  • 1,260
  • 5
  • 16
  • 27
  • 1
    If you need the output as sequences, don't add the Dense layer. Instead do `LSTM( X , return_sequences=True )`. This will return all the sequences. – Shubham Panchal Mar 01 '19 at 12:19
  • @ShubhamPanchal: What if I only need the 5 or 3 or 1 last output of the sequence? How can I specify this? – user3486308 Mar 01 '19 at 12:32
  • Are all your features numeric, and you want the next time-step prediction for each time-step of the 10? As far as I know, there isnt a way to only get part of the outputs only. You can just return sequences=true and only use part of it, however. – kylec123 Mar 01 '19 at 15:21
  • @kylec123: May you put a simple code for that? I don't know is it necessary to put a `Dense()` layer after `LSTM(10 , return_sequences = True)` layer to get the sequences or not? I also don't know how to get the complete sequence and split some part of it? – user3486308 Mar 01 '19 at 15:54
  • @kylec123: Yes, all of my data is numeric data. – user3486308 Mar 01 '19 at 15:55

1 Answers1

1

If you use:

LSTM(10 , return_sequences = True)

you will only get as output something of size (batch_size, num_timesteps, 10). The "10" for LSTM unit-size is a hyper-parameter you should be tuning. However, what you want is something the size of (batch_size, num_timesteps, num_features). This is why you want to put a Dense layer after the LSTM layer. That being said, you will also want to add a TimeDistributed wrapper around your Dense layer, so that a Dense layer is applied to every time-step (1):

TimeDistributed(Dense(num_features))

Now the output of the TD dense layer will be of size: (batch_size, num_timesteps, num_features), which I'm assuming is what you want. You will have to set up your Y data to be of the appropriate size for this to work. It will have to be of size (batch_size, num_timesteps, num_features). You mentioned that you are curious to know how to use just part of it. At prediction time, post training, you would call your model.predict and it will return something the size of (batch_size, num_timesteps, num_features). You can just use numpy array slicing to access what it is that you want.

kylec123
  • 161
  • 8