CNN model with both image data and pre-extracted features

I am trying to implement a CNN model to classify some images to their corresponding classes. Images are of size 64x64x3. My dataset consists of 25,000 images and also a CSV file consisting of 14 pre-extracted features like color, length etc.

I want to build a CNN model that make use of both the image data and the features for training and prediction. How can I implement such a model in Python with Keras.?

1 answer

  • answered 2018-04-14 15:52 Primusa

    I'm going to start out assuming that you can import the data without any issues, and you have already separated the x-data into Image and Features, and you have the y data as the labels of each image.

    You can use the keras functional api to have a neural network take multiple inputs.

    from keras.models import Model
    from keras.layers import Conv2D, Dense, Input, Embedding, multiply, Reshape, concatenate
    
    img = Input(shape=(64, 64, 3))
    features = Input(shape=(14,))
    embedded = Embedding(input_dim=14, output_dim=60*32)(features)
    embedded = Reshape(target_shape=(14, 60,32))(embedded)
    
    encoded = Conv2D(32, (3, 3), activation='relu')(img)
    encoded = Conv2D(32, (3, 3), activation='relu')(encoded)
    
    x = concatenate([embedded, encoded], axis=1)
    x = Dense(64, activation='relu')(x)
    x = Dense(64, activation='relu')(x)
    main_output = Dense(1, activation='sigmoid', name='main_output')(x)
    
    model = Model([img, features], [main_output])