Related questions

Tensorlfow Keras Negative dimension size caused by subtracting 2 from 1 for 'average_pooling2d' with input shapes: [?,1,1,32]

2018-01-13 17:44 Austin imported from Stackoverflow

Using Keras and Tensorflow with a 28x28x1 dataset. when I run the following code, it works just fine:

model = Sequential()

model.add(Convolution2D(8, 3, strides = 3, activation='relu', input_shape=(28,28,1),data_format = 'channels_last',padding='same'))

model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Convolution2D(16, 3, strides = 3, activation='relu',padding='same'))

model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Convolution2D(32, 3, strides = 3, activation='relu',padding='same'))
#model.add(AveragePooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

model.compile(loss='categorical_crossentropy',
              optimizer='adam',
              metrics=['accuracy'])

#Training the model in kreas
history = model.fit(train_dataset_kr,train_labels_kr,epochs = 500, validation_data = (valid_dataset_kr,vaild_labels_kr),batch_size = 256)
#model.train_on_batch(train_dataset_kr,train_labels_kr)

score = model.evaluate(test_dataset_kr, test_labels_kr, verbose=0)

as soon as I put in that final pooling2d(in this case, an AveragePooling2d), I get the following error:

ValueError: Negative dimension size caused by subtracting 2 from 1 for 'average_pooling2d_1/AvgPool' (op: 'AvgPool') with input shapes: [?,1,1,32].

Based on the size of my input file, I think I should be able to do 3 pooling2D. Any thoughts on what I might be doing wrong?

1 answer

  • answered 2018-01-13 17:44 Marcin Możejko

    You set way too big strides for your model. Let's check the output shapes of your network:

    model = Sequential()

model.add(Convolution2D(8, 3, strides = 3, activation='relu', input_shape=(28,28,1),data_format = 'channels_last',padding='same'))
# Shapes: (28, 28, 1) -> (10, 10, 8)
model.add(MaxPooling2D(pool_size=(2,2)))
# Shapes: (10, 10, 8) - > (5, 5, 8)
model.add(Dropout(0.25))

model.add(Convolution2D(16, 3, strides = 3, activation='relu',padding='same'))
# Shapes: (5, 5, 8) -> (2, 2, 16)
model.add(MaxPooling2D(pool_size=(2,2)))
model.add(Dropout(0.25))

model.add(Convolution2D(32, 3, strides = 3, activation='relu',padding='same'))
# Shapes: (2, 2, 16) -> (1, 1, 32)
#model.add(AveragePooling2D(pool_size=(2,2)))
# (1, 1, 32) - this shape is too small for pooling.
model.add(Dropout(0.25))

model.add(Flatten())
model.add(Dense(256, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(128, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(10, activation='softmax'))

    Are you sure you want to set your strides parameter to has value 3? It decreases output of each layer by a factor of 3.

See also questions close to this topic

  • how to solve 'range() integer end argument expected, got float'?

    I am getting the following error in the cnn_utils.py file when doing Coursera CNN example. I did everything I needed to convert the float to the int, but did not get the result. Please help me

    TypeError                                 Traceback (most recent call last)
<ipython-input-22-ddfc1f084c11> in <module>()
----> 1 _, _, parameters = model(X_train, Y_train, X_test, Y_test)

<ipython-input-12-a206d32574f3> in model(X_train, Y_train, X_test, Y_test, 
learning_rate, num_epochs, minibatch_size, print_cost)
 36             num_minibatches = int(m / minibatch_size) # number of 
minibatches of size minibatch_size in the train set
 37             seed = seed + 1
---> 38             minibatches = random_mini_batches(X_train, Y_train, 
minibatch_size, seed)
 39 
 40             for minibatch in minibatches:

/root/sharedfolder/cnn_utils.py in random_mini_batches(X, Y, mini_batch_size, seed)
 49     # Step 2: Partition (shuffled_X, shuffled_Y). Minus the end case.
 50     num_complete_minibatches = math.floor(m/mini_batch_size) # number of 
mini batches of size mini_batch_size in your partitionning
---> 51     for k in range(0, num_complete_minibatches):
 52         mini_batch_X = shuffled_X[k * mini_batch_size : k * 
mini_batch_size + mini_batch_size,:,:,:]
 53         mini_batch_Y = shuffled_Y[k * mini_batch_size : k * 
 mini_batch_size + mini_batch_size,:]

TypeError: range() integer end argument expected, got float.
  • Python how to loop multiple variables and different steps parallel?

    How to loop multiple variables and different steps parallel?

    Like, in c++, for(int i=0, j=n-1; i<n && j>=0; i++, j--).

  • How to split a big dataset into train, validation and testing sets

    I have a dataset with 30 classes, each class have different idx. I want to split this dataset into 70, 20, and 10%, train, validation and test sets respectively in python. can you please suggest me an idea how to write a code. I am new to coading.

  • how to run idle with tensorflow in virtual environment

    I have installed Tensorflow in a virtual environment in Ubuntu. And I have installed python 2.7 and IDLE. Whenever I am going to use Tensorflow I am activating with the command

      source ~/tensorflow/bin/activate

    I want to use IDLE to run my Tensorflow and Keras program. But I cant run with this setup. please help me to run the idle with Tensorflow (virtual)

  • tensorflow object detection API error to train

    I am using UBUNTU 16.04 and I have installer tensorflow and now i am trying to install the object detection API. I have cloned the API from git so it gave me a repository with the following tree: models/research/ From the research repo, i have following all the steps from the official git and i could run the test. Then after that i did:

    `sudo python setup.py install`

    I have also compiled from sources the protoc compiler to have the latest version: 3.5.1

    I have collected my data and created the tf record and put them in a data repository. I put the config file in a training repository. And i copy th data, training and model file repository into the models/researches/object-detection/

    then i tried to run the train.py script in order to and train with my own data but i get the following errors and i am really lost and maybe i did something wrong. Does someone has any idea?

    `python train.py --logtostderr --train_dir=training/ --pipeline_config_path=training/ssd_mobilenet_v1_coco.config 
/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
  from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Traceback (most recent call last):
  File "train.py", line 167, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "train.py", line 92, in main
    FLAGS.pipeline_config_path)
  File "/home/jimmy/Desktop/ObjectDetecAPI/models/research/object_detection/utils/config_util.py", line 92, in get_configs_from_pipeline_file
    proto_str = f.read()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 120, in read
    self._preread_check()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 80, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512, status)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: training/ssd_mobilenet_v1_coco.config; No such file or directory
'
   `/usr/local/lib/python2.7/dist-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type.
  from ._conv import register_converters as _register_converters
WARNING:tensorflow:From /usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Traceback (most recent call last):
  File "train.py", line 167, in <module>
    tf.app.run()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/platform/app.py", line 126, in run
    _sys.exit(main(argv))
  File "train.py", line 92, in main
    FLAGS.pipeline_config_path)
  File "/home/jimmy/Desktop/ObjectDetecAPI/models/research/object_detection/utils/config_util.py", line 92, in get_configs_from_pipeline_file
    proto_str = f.read()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 120, in read
    self._preread_check()
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/lib/io/file_io.py", line 80, in _preread_check
    compat.as_bytes(self.__name), 1024 * 512, status)
  File "/usr/local/lib/python2.7/dist-packages/tensorflow/python/framework/errors_impl.py", line 516, in __exit__
    c_api.TF_GetCode(self.status.status))
tensorflow.python.framework.errors_impl.NotFoundError: training/ssd_mobilenet_v1_coco.config; No such file or directory`

    But i am pretty sure that i have a training repository with a config file inside.

  • Image Classification - Tensorflow

    I am building an image classifier for 10 categories using tensorflow. My test accuracy is only around 25%. My parameters are: params = {"input_shape":[200,300,3], "conv_layers": 5, "filters":[40,50,60,60,60], "kernel_size":[5,5], "hidden_units": 9000, "drop_rate":0.4, "n_classes":10} and Relu activation function. I have 1000 training images per category.(300x200) what shall i do to get more accuracy? what alteration can be made in the params?How many epoch will be efficient?

  • ROC Curve Sklearn

    I've already seen other questions that address the issue that python scikit-learn's roc_curve function might return numbers of values a lot less than the number of data points and I know that this happens when there are a small number of unique values in the probability values.

    The following is the output of the first iteration of the for loop shown below:

    y_test: [0. 1. 0. 1. 0. 1. 0. 1. 0. 0. 1. 0. 0. 1. 0. 0. 0. 1. 0. 0. 0. 1. 0. 1. 1. 1. 0. 1. 0. 0. 0. 0. 0. 0. 0. 1. 0. 0. 1. 0. 1. 0. 0. 1.] probas: [0.97980869 0.61031697 0.9463976 0.07607395 0.93956894 0.06914656 0.64741115 0.07618758 0.95895803 0.83249766 0.13942336 0.7326476 0.93728438 0.07894027 0.97504296 0.92879864 0.93744224 0.21646299 0.95141726 0.92728865 0.97493415 0.07854641 0.95159664 0.36212405 0.21415855 0.10376292 0.95303641 0.11629533 0.93807975 0.7540189 0.93019584 0.94054764 0.93755026 0.93893753 0.95637685 0.10910955 0.96091857 0.95273078 0.61031697 0.9745807 0.11621697 0.97879922 0.96512002 0.09424992]

    Code:

    for i in range(0, 2):
    print("y_test: ", y_test[:, 1])
    print("probas: ", probas_[:, i])
    fpr[i], tpr[i], _ = roc_curve(y_test[:, 1], probas_[:, i], pos_label=1)
    roc_auc[i] = auc(fpr[i], tpr[i])

print("fpr", fpr)
print("tpr", tpr)
print("roc", roc_auc)

    Result:

    fpr {0: array([0., 0., 0., 1.]), 1: array([0.        , 0.03571429, 1.        , 1.        , 1.        ]), 'micro': array([0.        , 0.02272727, 0.95454545, 0.95454545, 1.        ,
           1.        ])}

tpr {0: array([0.0625, 0.875 , 1.    , 1.    ]), 1: array([0.   , 0.   , 0.   , 0.125, 1.   ]), 'micro': array([0.        , 0.        , 0.        , 0.04545455, 0.04545455,
           1.        ])}

roc {0: 1.0, 1: 0.0, 'micro': 0.002066115702479337}

    There are FOUR points for both FPR and TPR!!! Why is happening?

    Thank you!

  • Cannot run load_model (keras) on raspberry pi 3

    I'm facing an error when trying to load the model 'small_model.h5' The code is as follow

    small_model = load_model('small_model.h5')

    The error message is as follow

    AttributeError module 'tensorflow' has no attribute 'global_variables'

    I've tried to upgrade the tensorflow and keras on the board but it didn't help. Also, I've tried another way of reading the file by using

    small_model = 'small_model.h5'
small_modl = h5py.File(small_model,'r')

    and didn't help

    Any idea how to solve this? Thank you in advance.

  • What is the best Pythonic way to convert streaming speech to text

    I have successfully implemented a small project with the help of google API to convert recorded speech to text that also includes SpeechRecognition Python module. Now I wanted to start a project wherein I have to convert live streaming speech to text. But I am not sure how to go about it. Which Python module to use? sample example. This time I do not wish to use google API.

    Can somebody guide me please

  • How can a specific model be cleared from the memory?

    I created several Keras models (Model_1, Model_2...Model_N) in one code. I would like to clear only one specified model (e.g. Model_1).

    I guess K.clear_session(), which will clear all models from the memory, is not useful in this case. Is there any solution?

    Thank you in advance.

  • Second derivative in Keras

    For a custom loss function for a NN I need to use the equation enter image description here. The function u, given a pair (t,x), is the the output of my NN. Problem is I'm stuck at how to compute the second derivative using K.gradient (K being the TensorFlow backend):

    def custom_loss(input_tensor, output_tensor):
    def loss(y_true, y_pred):

        # so far, I can only get this right, naturally:            
        gradient = K.gradients(output_tensor, input_tensor)

        # here I'm falling badly:

        # d_t = K.gradients(output_tensor, input_tensor)[0]
        # dd_x = K.gradient(K.gradients(output_tensor, input_tensor),
        #                   input_tensor[1])

        return gradient # obviously not useful, just for it to work
    return loss

    All my attemps, based on Input(shape=(2,)), were variations of the commented lines in the snippet above, mainly trying to find the right indexation of the resulting tensor.

    Sure enough I lack knowledge of how exactly tensors work. By the way, I know in TensorFlow itself I could simply use tf.hessian, but I noticed it's just not present when using TF as a backend.