neuralnetwork how to choose the best model
I am using nftool to train my data (90 piece of data), and my training algorithm is "Bayesian Regularization" I am going to use different neurons to compare and find which one is the best model. Is there any appropriate step I need to follow? Do I need to retrain again and again until getting the good performance of model for each neurons?
See also questions close to this topic

combinations totaling to sum in 2d
The task is to find all
N×N
matricesM
with integer values, when the sums along the cols and rows are fixed.For
N=3
this means: Find the matrixM
m11 m12 m13 m21 m22 m23 m31 m32 m33
so that
m11 + m12 + m13 = r1 m21 + m22 + m23 = r2 m31 + m32 + m33 = r3
and
m11 + m21 + m31 = c1 m12 + m22 + m32 = c2 m13 + m23 + m33 = c3
holds.
e.g.: For
r1=3, r2=6, r3=9, c1=5, c2=6, c3=7
one solution would be:
0 1 2 1 2 3 4 3 2
The only algorithm I came up is to calculate straightforward all possible partitions for the rows, start with the first partition (
p11
,p12
,...
,p1N
) of the first column, select all compatible partitions (p21
,p22
,...
,p2N
) of the second row by checkingp11+p21 <= c1
,...
,p1N+p2N <= cN
, and proceed with the third row. If one wants to visit all possible solutions this scales obviously very badly withN
.The 1D problem  the partition of a number
A
intoN
parts  is solved by a number of algorithms. One basic idea is to calculate all possibilities to split a string ofA
'+' by(N1)
'' and count the numbers of+
between the
(e.g.:++++++
yields3,2,1,0
). I wonder if it is possible to extend this ansatz to the 2D problem. Also, I suspect that it might be possible to calculate a solution based on the preceding solution and iterate over all solutions. 
sequence of assigning weights to multiclass classification task using libsvm?
I have 4 classes with labels 1,6,7,8 (the labels can be from 1 to 4 but since I have more than 4 classes which will be used for other classification tasks so I am strict to these). I computed the weights by setting the ratio of penalties for different classes to the inverse ratio of the training class sizes (http://ieeexplore.ieee.org/abstract/document/1527706/).
w(weight) of label1 =3.0752, w of label6 = 1.9917, w of label7= 1, weight of label8 =38.006; model1 = svmtrain(train_label, train_featureN, 'w0 3.0752 w1 1.9917 w2 1 w3 38.006'); model2 = svmtrain(train_label, train_featureN, 'w1 3.0752 w6 1.9917 w7 1 w8 38.006');
what would by the appropriate sequence for the said labels and weights (in matlab)? The problem is that I am getting different accuracy value for model1 and model2?
This is also confusing as mentioned in http://www.csie.ntu.edu.tw/~cjlin/libsvm/faq.html#f430 that "Internally class labels are ordered by their first occurrence in the training set. For a kclass data, internally labels are 0, ..., k1, and each twoclass SVM considers pair (i, j) with i < j. Then class i is treated as positive (+1) and j as negative (1). For example, if the data set has labels +5/+10 and +10 appears first, then internally the +5 versus +10 SVM problem has +10 as positive (+1) and +5 as negative (1)."

What is wrong with my loop code?
I wrote a loop code to extract my table files names into a string or an array and meanwhile collect the data into arrays. But I found my code went wrong as only one file was read and repeated in the loop over and over again. I had no idea where my code is wrong and it already took me several hours to look for the problem. Could somebody help me?
DataCircle = dir('*circle.xls'); MeanAreaCircle = []; ColonyNumCircle = []; PlateNameCircle = []; for zz = 1:numel(DataCircle) basefilenamedata1 = DataCircle(w).name; % generate the base name DataName1 = regexprep(basefilenamedata1,'circle.xls',''); %replace part of the name and the extension PlateNameCircle = [PlateNameCircle DataName1]; % collect the file name into a string T1 = readtable(basefilenamedata1); % read data in MeanAreaCircle = [MeanAreaCircle mean(T1.area)]; % collect the mean for area end
What I got is like this, which is wrong:
>> PlateNameCircle PlateNameCircle ='IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813IMG_0813' >> MeanAreaCircle MeanAreaCircle = 1.0e+03 * 6.4152 6.4152 6.4152 6.4152 6.4152 6.4152 6.4152 6.4152 6.4152 6.4152
My input file list:
IMG_0809CC.xls IMG_0809circle.xls IMG_0810CC.xls IMG_0810circle.xls IMG_0812CC.xls IMG_0812circle.xls IMG_0813CC.xls IMG_0813circle.xls
What I want is a column or a character array or a string like this:
PlateNameCircle = 'IMG_0809' 'IMG_0810' 'IMG_0811' 'IMG_0812' 'IMG_0813'

Javascript Neural Net returns 0.66ish and 0 on XOR
So my issue is that after running the XOR problem trough my code 10 000 times it will return 0.66ish for all three inputs except [0,0], this returns 0 (as it should).
I have set it up with 3 layers: input(2) > hidden(8) > output(1). I generate random "weights" and pass the "input" trough each layer multiplying it with the weights and feeding it trough a TAHN function. After this I backpropagate the output layer by calculating the "error" and multiply it by the squared "output" of the layer (to generate "gamma", this is done, on the advice of a friend). I then multiply the "gamma" with the "input" to create "weightsDelta". A similar process is then applied to the following hidden layer (see backProprHidden()). And the layers in turn have their weights updated (updateWeights()) subtracting "weightsDelta" muliplied by a 0.00333 (learning rate)
This should in my opinion (which seems to be shit worth if you ask javascript<3..) generate the needed weights to solve XOR.
The code is run in tests.js. Here's a "flowchart".
Create layers > Create net obj with layers > run XOR trough (compute > feed > backprop > update) > Finally test compute to again see if it returns correct answers.
This is not working and i think it is a matter of the backprop or my javascript skills overall. I have poor knowledge of scopes and this is probably pretty imperative for a task such as this.
Anyhow heres the code!
Layers code:
function layer(i_c, o_c){ this.output_count= o_c; this.input_count = i_c; //Feed forward vars this.output = new Array(this.output_count) this.weights = new Array(this.output_count); this.input = new Array(this.input_count) // //Backprop vars console.log("INITIAL OC",this.output_count) this.gamma = new Array(this.output_Count) this.error = new Array(this.output_Count) this.weightsDelta = new Array(this.output_count); // //Generate weights for (var i = 0; i < this.output_count; i++){ this.weights[i] = new Array(this.input_count); this.weightsDelta[i] = new Array(this.input_count); for(var j= 0; j< this.input_count; j++){ this.weights[i][j] = Math.random() 0.5; console.log("Generating weights: " + this.weights[i][j]) } } this.square = function(val) { return 1  (val *val) } this.feedForward = function(input) { this.input = input; for (var i = 0; i < this.output_count; i++) { this.output[i] = 0; for (var j = 0; j < this.input_count; j++) { this.output[i] += (this.input[j] * this.weights[i][j]); } this.output[i] = Math.tanh(this.output[i]); } } this.backPropOut = function(expected) { this.expected = expected; for(var i = 0; i < this.output_count; i++) { this.error[i] = (this.output[i]  this.expected[i]); } for(var j = 0; j < this.output_count; j++) { this.gamma[j] = this.error[j] * this.square(this.output[j]) //Square } for (var k = 0; k < this.output_count; k++) { //console.log("calc delta") for (var l = 0; l < this.input_count; l++) { this.weightsDelta[k][l] = this.gamma[k] * this.input[l]; } } } this.backPropHidden = function(gammaF, weightsF) { this.gammaF = gammaF; this.weightsF = weightsF; //console.log(weightsF[0].length) for (var i = 0; i < this.output_count; i++) { this.gamma[i] = 0; for (var j = 0; j < this.gammaF.length; j++) { this.gamma[i] += this.gammaF[j] * this.weightsF[j][i]; } this.gamma[i] *= this.square(this.output[i]); //Square } for (var i = 0; i < this.output_count; i++) { for (var j = 0; j < this.input_count; j++) { //console.log("Creating hidden layer deltas") this.weightsDelta[i][j] = this.gamma[i] * this.input[j]; } } } this.updateWeights = function() { for (var i = 0; i < this.output_count; i++) { for (var j = 0; j < this.input_count; j++) { this.weights[i][j] = this.weightsDelta[i][j] * 0.00333; //Lrning rate ******REQUIRED***** } } } }
Neural net code:
function NN(layer_objs) { this.layer_objs = layer_objs; //Functions this.compute = function(input) { this.layer_objs[0].feedForward(input); for (var i = 1; i < this.layer_objs.length; i++) { this.layer_objs[i].feedForward(this.layer_objs[i  1].output); } //console.log("Output: ", this.layer_objs[this.layer_objs.length1].output); return this.layer_objs[this.layer_objs.length 1].output; } this.backProp = function(expected){ this.expected = expected; for (var i = this.layer_objs.length 1; i >= 0; i) { if (i == this.layer_objs.length 1) { //console.log("Expected__________________",this.expected) this.layer_objs[i].backPropOut(this.expected); } else { this.layer_objs[i].backPropHidden(this.layer_objs[i +1].gamma, this.layer_objs[i + 1].weights); } } for (var i = 0; i < this.layer_objs.length; i++) { this.layer_objs[i].updateWeights(); } } }
Test code:
//var l = layer.buildLayers([2,8,1]) var l = [new layer(2,8), new layer(8,1),new layer(1,1)] console.log(l) var network = new NN(l) console.log(network) for (var i = 0; i < 10000; i++) { console.log(i) network.compute([ 1, 0 ]); network.backProp([1]); network.compute([ 0, 0 ]); network.backProp([0]); network.compute([ 0, 1 ]); network.backProp([1]); network.compute([1, 1 ]); network.backProp([0]); } console.log("") console.log(network.compute([1,0])[0]) console.log(network.compute([1,1])[0]) console.log(network.compute([0,0])[0]) console.log(network.compute([0,5])[0])
And finally and index file for those who feel like trying it out!
<!DOCTYPE html> <html> <head> <meta name="viewport" width=devicewidth, initialscale=1.0, maximumscale=1.0, userscalable=0> <style> body {padding: 0; margin: 0;} </style> <script src="../p5.min.js"></script> <script src="../addons/p5.dom.min.js"></script> <script src="../addons/p5.sound.min.js"></script> <script src="sketch.js"></script> <script src="nn.js"></script> <script src="layer.js"></script> <script src="gpu.js"></script> <script src="tests.js"></script> </head> <body> </body> </html>
You'll find the output in the console if you open the index file in your browser! Thanks in advance!

Evaluation of Regression Neural Network
Hej,
I am trying to write a small program to solve a Regression problem. My dataset is hereby 4 random x (x1,x2,x3 and x4) and 1 y value. One of the rows looks like this:
0.634585 0.552366 0.873447 0.196890 8.75
I know want to predict the yvalue as close as possible so after the training I would like to evaluate how good my model is by showing the loss. Unfortunately I always receive
Training cost= nan
The most important lines of could would be:
X_data = tf.placeholder(shape=[None, 4], dtype=tf.float32) y_target = tf.placeholder(shape=[None, 1], dtype=tf.float32) # Input neurons : 4 # Hidden neurons : 2 x 8 # Output neurons : 3 hidden_layer_nodes = 8 w1 = tf.Variable(tf.random_normal(shape=[4,hidden_layer_nodes])) # Inputs > Hidden Layer1 b1 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes])) # First Bias w2 = tf.Variable(tf.random_normal(shape=[hidden_layer_nodes,1])) # Hidden layer2 > Outputs b2 = tf.Variable(tf.random_normal(shape=[1])) # Third Bias hidden_output = tf.nn.relu(tf.add(tf.matmul(X_data, w1), b1)) final_output = tf.nn.relu(tf.add(tf.matmul(hidden_output, w2), b2)) loss = tf.reduce_mean(tf.reduce_sum(y_target * tf.log(final_output), axis=0)) optimizer = tf.train.GradientDescentOptimizer(learning_rate=0.001) train = optimizer.minimize(loss) init = tf.global_variables_initializer() steps = 10000 with tf.Session() as sess: sess.run(init) for i in range(steps): sess.run(train,feed_dict={X_data:X_train,y_target:y_train}) # PRINT OUT A MESSAGE EVERY 100 STEPS if i%500 == 0: print('Currently on step {}'.format(i)) training_cost = sess.run(loss, feed_dict={X_data:X_test,y_target:y_test}) print("Training cost=", training_cost)
Maybe someone knows where my mistake is or even better, how to constantly show the error during my training :) I know how this is done with the tf.estimator, but not without. If you need the dataset, let me know.
Cheers!

Practical difference between deconvolution and resize followed by convolution for upscaling
I need to go (upsample) from a layer
input = HxWxn1
, where H: height, W:width and n1: number of filters, to a layeroutput = 2Hx2Wxn2
, where 2H = 2*Height etc, andn2=n1/2
: number of new filters. One way of achieving this is by using transposed convolution operators. However, it is known that deconvolution (transposed convolution) operators can lead to the checkerboard artifacts. One way to overcome this problem is to perform resize and then apply a convolution map. E.g.output = transpose_conv2d(input,n2,kernel!=(1,1),stride!=1)
vs
output = resize(input) # e.g. bilinear interpolation output = conv2d(output,n2,kernel=(1,1),stride=1)
Obviously, in the second case, we are just changing the number of filters, and we are "not learning" any feature (we do not summarize information from
input
layer). But this can be solved by another convolution with kernel size,kernel!= 1
. E.g.output = resize(input) output = conv2d(output,n2,kernel=(1,1),stride=1) # appropriate padding output = conv2d(output, n2, kernel != (1,1), ...)
Are there any practical differences (besides computational complexity) between the two approaches for upsampling? I understand that the latter solves the problem of checkerboard artifacts.