neuralnetwork how to choose the best model
I am using nftool to train my data (90 piece of data), and my training algorithm is "Bayesian Regularization" I am going to use different neurons to compare and find which one is the best model. Is there any appropriate step I need to follow? Do I need to retrain again and again until getting the good performance of model for each neurons?
See also questions close to this topic

Matlab anonymous callback function arguments
Following this example of a Model View Controller GUI in Matlab, I have a question regarding the anonymous function callback input arguments
Here is the a view function that creates the gui handles and passes them as an input argument to the onChanged callback function.
function handles = View_TimeDomain(m) %VIEW a GUI representation of the signal model % build the GUI handles = initGUI(); onChangedF(handles, m); % populate with initial values % observe on model changes and update view accordingly % (tie listener to model object lifecycle) addlistener(m, 'f', 'PostSet', ... @(o,e) onChangedF(handles,e.AffectedObject)); end
The first thing that I don't quite understand is that according to the Matlab documentation the first argument must be the source of the event and the second arument must be the event data (Matlab doc1, Matlab doc2) but in this case it is
handles
. When the event is triggered, the following onChangedF function is called as expected.function onChangedF(handles, model) % respond to model changes by updating view if ~ishghandle(handles.fig), return, end set(handles.line, 'XData',model.t, 'YData',model.data) set(handles.slider, 'Value',model.f); end
However, in this case handles is the struct containing the handles defined using initGui() and not the event source.
I guess this stems from the definition of the anonymous function:
@(o,e) onChangedF(handles, e.AffectedObject)
Am I correct, assuming that
o
is the source which is not used in theonChangedF
function input. Can someone explain why this syntax of an anonymous function is valid? I thought thato
needs to be an argument of this specific callback function as well. Something like this:@(o,e) onChangedF(o, handles, e.AffectedObject)
where additional arguments are appended (at the end) of the function. And then just avoid this unused argument using ~:
function onChangedF(~, handles, model) % respond to model changes by updating view if ~ishghandle(handles.fig), return, end set(handles.line, 'XData',model.t, 'YData',model.data) set(handles.slider, 'Value',model.f); end

Reshape structure into a matrix without for loop
I have a structure (data) which consist of 322 cells a 296(features)*2000(timepoints). I want a matrix per timepoint which consists of trials^features^timepoints (322 *296*2000). What I am currently doing and what also works fine is using a forloop:
for k=1:size(data.trial{1,1},2) for i= 1:length(data.trialinfo) between=data.trial{1,i}'; data(i,:,k)=between(k,:); end end
Can anyone think of a faster way to do that? Because it takes ages as the matrix increases.
Thanks!
Carlos

Matlab HMM Murphys Toolbox for classification
At first i have to mention that I've read and watched different tutorials about the implementation for murphys toolbox and also have some backround to the topic of HMM. But all application examples handel either the subject to predict the weather or voice recognition. Neither of the two cases is similar to my application.
I have to make a classification for my master's thesis. In that case the model should distinguish between two different driving maneuvers. As input variables I have 20 differnent vehicle specific quantities for every maneuver from several test persons. This means I want to train the model with train data and made a classification for test people. I know that for each state a separate model must be trained, in my issue two. In this case i would set:
O=2; %number of states (maneuver1, maneuever2) Q=20; %number of discrete observation (all 20 vehicle specific quantities) I know the meaning of prior0, transmat0 and obsmat0, but did i need these one if i want to train the model with training data? And if yes how I get these Informations?
If read that the murphys toolbox offers different subdivison like dhmm, ghmm and mhmm. Which of these is most appropriate or where is the difference.
The em algorithm is used to train the model, how do I put my data record here? Is it necessary to make the variable prior, transmat and obsmat random, as seen in different tutorials?
As you can see, I have many questions, which have not yet been answered, hopefully you can help me.
Best regards

How to use ImageDataGenerator with my own set of images for image augmentation
From here it shows how to do image augmentation with an existing image set,
cifar10
.How can I create an
ImageDataGenerator
with my own set of images? 
Difference between following keras and tensorflow code in implementing GRU?
I am relatively new to both keras and tensorflow. I have recently switched to keras from tensorflow for better control.
I am trying to implement GRU (or lstm) on a sequential data. These are codes I used to impelement in tensorflow and keras respectively.. But both are giving me different results. Tensorflow implementation is giving good results than keras implementation. Aren't these two trying to do the same??
Tensorflow:
def RNN (x, weights, biases, pkeep):
# Unstack to get a list of 'n_steps' tensors of shape (batch_size, n_input) x = tf.unstack(x, n_steps, 1) cells = [rnn.GRUCell(n_hidden) for _ in range(3)] dropcells = [rnn.DropoutWrapper(cell,input_keep_prob=pkeep) for cell in cells] rnn_cell = rnn.MultiRNNCell(dropcells) rnn_cell = rnn.DropoutWrapper(rnn_cell, output_keep_prob=pkeep) # Get lstm cell output outputs, states = rnn.static_rnn(rnn_cell, x, dtype=tf.float32) # Linear activation, using rnn inner loop last output return tf.matmul(outputs[1], weights['out']) + biases['out']
Keras:
def RNN_keras(x, weights, biases, pkeep):
#x = Dropout(pkeep)(x) y_k = GRU(n_hidden, return_sequences=True,input_shape=(n_steps, n_input))(x) #y_k = BatchNormalization()(y_k) y_k = Dropout(dropout_pkeep)(y_k) y_k = GRU(n_hidden, return_sequences=True)(y_k) #y_k = BatchNormalization()(y_k) y_k = Dropout(dropout_pkeep)(y_k) y_k = GRU(n_hidden)(y_k) #y_k = BatchNormalization()(y_k) y_k = Dropout(dropout_pkeep)(y_k) # Linear activation, using rnn inner loop last output return tf.matmul(y_k, weights['out']) + biases['out']
Thanks in advance :)
Note: pkeep is a tensor and I am passing the same value dropout_pkeep to pkeep while running the session!

How to overcome overfitting in convolutional neural network when nothing helps?
I'm training a convolutional neural network with siamese architecture and constrastive loss function for face verification task. And I'm faced with a huge difference in training and validation accuracy starting from literally first three or five epochs. When training accuracy reaches 95% I have ~65% validation accuracy. It is fluctuating somewhere near 70% but never reaches this number. these are training and validation accuracy plotted on one chart
So to avoid this I tried a range of standard techniques when it comes to overfitting, but before listing them here I should say that none of them really changes the picture. The gap between training and validation accuracy stays the same. So I used:
 L1 regularization with lambda varying from 0.0001 to 10000.0
 L2 regularization with lambda varying from 0.0001 to 10000.0
 Dropout with rate from 0.2 to 0.8
 Data augmentation techniques (rotation, shifting, zooming)
 Removing fully connected layers except last layer.
Nothing of these really help, so I appreciate any advises from you guys. And some information about the network itself. I'm using tensorflow. This is how the model itself look like:
net = tf.layers.conv2d( inputs, kernel_size=(7, 7), filters=15, strides=1, activation=tf.nn.relu, kernel_initializer=w_init, kernel_regularizer=reg) # 15 x 58 x 58 net = tf.layers.max_pooling2d(net, pool_size=(2, 2), strides=2) # 15 x 29 x 29 net = tf.layers.conv2d( net, kernel_size=(6, 6), filters=45, strides=1, activation=tf.nn.relu, kernel_initializer=w_init, kernel_regularizer=reg) # 45 x 24 x 24 net = tf.layers.max_pooling2d(net, pool_size=(4, 4), strides=4) # 45 x 6 x 6 net = tf.layers.conv2d( net, kernel_size=(6, 6), filters=256, strides=1, activation=tf.nn.relu, kernel_initializer=w_init, kernel_regularizer=reg) # 256 x 1 x 1 net = tf.reshape(net, [1, 256]) net = tf.layers.dense(net, units=512, activation=tf.nn.relu, kernel_regularizer=reg, kernel_initializer=w_init) net = tf.layers.dropout(net, rate=0.2) # net = tf.layers.dense(net, units=256, activation=tf.nn.relu, kernel_regularizer=reg, kernel_initializer=w_init) # net = tf.layers.dropout(net, rate=0.75) return tf.layers.dense(net, units=embedding_size, activation=tf.nn.relu, kernel_initializer=w_init)
This is how loss function is implemented:
def contrastive_loss(out1, out2, labels, margin): distance = compute_euclidian_distance_square(out1, out2) positive_part = labels * distance negative_part = (1  labels) * tf.maximum(tf.square(margin)  distance, 0.0) return tf.reduce_mean(positive_part + negative_part) / 2
This is how I get and augment data (I'm using LFW dataset):
ROTATIONS_RANGE = range(1, 25) SHIFTS_RANGE = range(1, 18) ZOOM_RANGE = (1.05, 1.075, 1.1, 1.125, 1.15, 1.175, 1.2, 1.225, 1.25, 1.275, 1.3, 1.325, 1.35, 1.375, 1.4) IMG_SLICE = (slice(0, 64), slice(0, 64)) def pad_img(img): return np.pad(img, ((0, 2), (0, 17)), mode='constant') def get_data(rotation=False, shifting=False, zooming=False): train_data = fetch_lfw_pairs(subset='train') test_data = fetch_lfw_pairs(subset='test') x1s_trn, x2s_trn, ys_trn, x1s_vld, x2s_vld = [], [], [], [], [] for (pair, y) in zip(train_data.pairs, train_data.target): img1, img2 = pad_img(pair[0]), pad_img(pair[1]) x1s_trn.append(img1) x2s_trn.append(img2) ys_trn.append(y) if rotation: for angle in ROTATIONS_RANGE: x1s_trn.append(np.asarray(rotate(img1, angle))[IMG_SLICE]) x2s_trn.append(np.asarray(rotate(img2, angle))[IMG_SLICE]) ys_trn.append(y) x1s_trn.append(np.asarray(rotate(img1, angle))[IMG_SLICE]) x2s_trn.append(np.asarray(rotate(img2, angle))[IMG_SLICE]) ys_trn.append(y) if shifting: for pixels_to_shift in SHIFTS_RANGE: x1s_trn.append(shift(img1, pixels_to_shift)) x2s_trn.append(shift(img2, pixels_to_shift)) ys_trn.append(y) x1s_trn.append(shift(img1, pixels_to_shift)) x2s_trn.append(shift(img2, pixels_to_shift)) ys_trn.append(y) if zooming: for zm in ZOOM_RANGE: x1s_trn.append(np.asarray(zoom(img1, zm))[IMG_SLICE]) x2s_trn.append(np.asarray(zoom(img2, zm))[IMG_SLICE]) ys_trn.append(y) for (img1, img2) in test_data.pairs: x1s_vld.append(pad_img(img1)) x2s_vld.append(pad_img(img2)) return ( np.array(x1s_trn), np.array(x2s_trn), np.array(ys_trn), np.array(x1s_vld), np.array(x2s_vld), np.array(test_data.target) )
Thanks all!