2

I am trying to train an LSTM model using Keras functional API. My training data is of shape:

>>> data.shape()
(100000,variable_sequence_lengths,295)

where 100000 corresponds to the number of instances (the whole number of sequences) and 295 denotes the number of features in each element of a given sequence.

I am getting errors regarding the shape of the input data. How to define the shape of the data in the input layer and the subsequent one (LSTM) considering variable sequence lengths?

from keras.layers import Input, Dense, concatenate, LSTM
from keras.models import Model
import numpy as np

inputs = Input(shape=(x,y,z))
x=LSTM(128, return_sequences=True, input_shape=(a,b,c))(inputs)
....

What values should x, y, z and a, b, c take?

Yacine
  • 143
  • 1
  • 5

3 Answers3

3

Model specification is per sample, number of samples should not be included in the dimensions, i.e. (a, b, c) or (sample_size, timesteps, dimension) should be changed to (timesteps, dimension). Also, for variable-length sequences, timesteps should be None. That is, (timesteps, dimension) should be (None, 295). The same goes for (x, y, z).

Please check out this post for feeding variable-length multi-dimensional sequences to Keras LSTM.

Esmailian
  • 9,147
  • 2
  • 31
  • 47
0

In Keras functional API, you can use only one Input function as mention in following. If you have used Input then do not mention input shape in LSTM layer.

from keras.layers import Input, Dense, concatenate, LSTM
from keras.models import Model
import numpy as np

# 64 = batch size
# 128 = sequence length
# 295 = number of features

inputs = Input(shape = (64, 128, 295))
x = LSTM(128, return_sequences = True)(inputs)
Swapnil Pote
  • 401
  • 3
  • 9
  • In your answer, you assume that the sequences have the same length. Indeed, it is not the case in my example. – Yacine May 20 '19 at 12:16
0

I had wondered this my self in the past. But from everything that I am able to determine, it is not possible to actually have different input sizes to the same network. For example if you have a sequence of length 300 and one of length 500, to use them as input in the same network, you will have to either extend the length of the smaller sequence by padding the ends of it some how, or break up the longer one into smaller sub-sequences. If you absolutely have to use different length inputs, and can't use either of these methods, you might need to switch to a Hidden Markov Model, which uses transition probabilities to make predictions, can use prior probabilities for current predictions, and will make predictions on any length of sequence that is input to it. This is still the de-facto standard for most genetics tools on long sequences.

However another method that you might try, especially if you are interested in sequence labeling rather than just sequence classification, is to use the natural language processing style of having a target with a set context window around it. You run the sub-sequence through the network, and then scoot the window over one, with its new context window. Very time consuming, but has the possibility to be quite effective.

superhelicase
  • 23
  • 5