training vs validation accuracy and loss output error
Can anyone help me interpret these graphs.
I have been training some datasets over SENet for image classification (tensorLayer).
I started with cifar10 and got some satisfying results with it I use data augmentation for all my experiments.
Next i tried training it with Caltech256 and the results were pretty unusual Maybe due to less number of images per class(3050)
Finally i used TinyImagenet with enough training samples per class(500) but the results are still looking odd somewhat similar to caltech256 results
Is that supposed to be a problem with the data??
See also questions close to this topic

need advice for best approach to specific machine learning method
What would be a good approach to a classification neural network that has a 3dimensional array of floats as an input and outputs to around 20 different classes?
it's worth mentioning that most of the data in these arrays will be unpopulated, as in, left at 0.00 while some specific cells will always have an input.

Training SVM with images of different categories
I am new to machine learning, my aim is to classify traffic/road signs using
SVM
.My question is: Can the training
data set
ofpositive
images contains different categories of images?ex: I want to classify those two type of images to turn
RIGHT
, is it possible ? would be the data still linearly separable? 
Deciding to the clustering algorithm for the dataset containing both categorical and numerical variables
I am a newbie in machine learning and trying to make a segmentation with clustering algorithms. However, Since my dataset has both categorical variables (such as gender, marital status, preferred social media platform etc) as well as numerical variables ( average expenditure, age, income etc.), I could not decide which algorithms worth to focus on. Which one should I try: fuzzy c means, kmedoids, or latent class to compare with kmeans++? which ones would yield better results for these type of mixed datasets?
Bonus question: Should I try to do clustering without dimensionality reduction? or should I use PCA or KPCA in any case to decrease dimensions? Also, how can I understand and interpret results without visualization if the dataset has more than 3 dimensions ?

adehabitat compana() doesn't work or returns lambda=NaN
I'm trying to do the
compositional
analysis of habitat use with thecompana()
function in the adehabitatHS package (I use adehabitat because I can't install adehabitatHS).Compana()
needs two matrices: one of habitat use and one of avaiable habitat.When I try to run the function it doesn't work (it never stops), so I have to abort the RStudio session.
I read that one problem could be the
0values
in some habitat types for some animals in the 'avaiable' matrix, whereas other animals have positive values for the same habitat. As done by other people, I replaced0values
with small values (0,001), rancompana
and it worked BUT the lambda values returned meNaN
.The problem is similar to the one found here
adehabitatHS compana test returns lambda = NaN?
They said they resolved using as 'used' habitat matrix the counts (integers) and not the proportions.
I tried also this approach, but never changed (it freezes when there are
0values
in the available matrix, or returnsNaN
value for Lambda if I replace 0 values wit small values).I checked all matrices and they are ok, so I'm getting crazy.
I have 6 animals and 21 habitat types. Can you resolve this BIG problem?

R  Looping over a list of metrics on a user defined model
I currently have a difference in difference model and would like to measure different metrics for the same model in an efficient manner.
For example, I have a data frame with columns for Miles Driven, Hours Worked, State, Group, Time.
Currently, have code where I copy and paste the model for each metric:
# Create DID models model_miles < lm(df$miles_driven ~ df$Group + df$Time + df$Group * df$Time, data = df) model_hours < lm(df$hours_worked ~ df$Group + df$Time + df$Group * df$Time, data = df) # Select results using clustered standard errors. The purpose of this is to # avoid making distributional assumptions about the errors in the models. results_miles < clubSandwich::coef_test(model_miles, vcov = "CR2", cluster = df$state, test = "Satterthwaite") results_hours < clubSandwich::coef_test(model_hours, vcov = "CR2", cluster = df$state, test = "Satterthwaite") results < data.table::rbindlist(list(results_miles, results_hours)) View(results)
I would like to somehow create a list of my metric names, and loop over this list using a user defined function, in order to make this process faster and more automated, but I haven't been able to get this to work correctly:
#list of metrics metrics < c("miles_driven", "hours_worked") udf < function(metric, dataframe){ # Create DID model model < lm(dataframe$metric ~ df$Group + dataframe$Time + dataframe$Group * df$Time, data = dataframe) # Select results using clustered standard errors. The purpose of this is to # avoid making distributional assumptions about the errors in the models. results_miles < clubSandwich::coef_test(model_miles, vcov = "CR2", cluster = dataframe$state, test = "Satterthwaite")[4,] View(results) } lapply(metrics, udf)
Any insight would be appreciated. Thanks!

Binomial test returns pvalue = 0
I am using binom.test and very occasionally find results that give a pvalue of
0
.An example is:
binom.test(x=700, n=3000, p=0.01, alternative = "greater")$p.val
x = number of successes, or a vector of length 2 giving the numbers of successes and failures, respectively
n = number of trials; ignored if x has length 2
p = hypothesized probability of successWhat can I do here? Is this an expected result?
 3D Mophable Model fitting algorithm suggestion

Tensorflow gradients are 0, weights are not updating
I'm trying to learn TensorFlow after using Keras for a while, and I'm trying to build a ConvNet for CIFAR10 classification. However, I think I misunderstand something in TensorFlow API, since the weights are not updating even in 1layer net model.
The code for the model is as follows:
num_epochs = 10 batch_size = 64 # Shape of mu and std is correct: (1, 32, 32, 3) mu = np.mean(X_train, axis=0, keepdims=True) sigma = np.std(X_train, axis=0, keepdims=True) # Placeholders for data & normalization # (normalisation does not help) data = tf.placeholder(np.float32, shape=(None, 32, 32, 3), name='data') labels = tf.placeholder(np.int32, shape=(None,), name='labels') data = (data  mu) / sigma # flatten flat = tf.reshape(data, shape=(1, 32 * 32 * 3)) dense1 = tf.layers.dense(inputs=flat, units=10) predictions = tf.nn.softmax(dense1) onehot_labels = tf.one_hot(indices=labels, depth=10) # Tried sparse_softmax_cross_entropy_with_logits as well loss = tf.losses.softmax_cross_entropy(onehot_labels=onehot_labels, logits=predictions) loss = tf.reduce_mean(loss) # Learning rate does not matter as the weights are not updating! optimizer = tf.train.AdamOptimizer(learning_rate=0.01).minimize(loss) loss_history = [] with tf.Session() as session: tf.global_variables_initializer().run() tf.local_variables_initializer().run() for epochs in range(10): print("Epoch:", epochs) # Load tiny batches for batch in iterate_minibatches(X_train.astype(np.float32)[:10], y_train[:10], 5): inputs, target = batch feed_dict = {data: inputs, labels: target} loss_val, _ = session.run([loss, optimizer], feed_dict=feed_dict) grads = tf.reduce_sum(tf.gradients(loss, dense1)[0]) grads = session.run(grads, {data: inputs, labels: target}) print("Loss:", loss_val, "Grads:", grads)
The code produces the following output:
Epoch: 0 Loss: 2.46115 Grads: 1.02031e17 Loss: 2.46041 Grads: 0.0 Epoch: 1 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 2 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 3 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 4 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 5 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 6 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 7 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 8 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0 Epoch: 9 Loss: 2.46115 Grads: 0.0 Loss: 2.26115 Grads: 0.0
It looks like the model probably resets its weights somehow or stops learning completely. I have also tried sparse softmax crossentropy loss, but nothing helps.

On creating the A matrix for the 8 point algorithm
So I've been trying to work on a program that utilizing points selected by the user, can utilize the 8 point algorithm to find the fundamental matrix and then from there can figure out intrinsic and extrinsic characteristics.
[X, Y] = ginput(1); Cl = X(1); Rl = Y(1); pl(cnt,:) = [Cl Rl 1]; [X, Y] = ginput(1); Cr = X(1); Rr = Y(1); pr(cnt,:) = [CrCOLS Rr 1];
So with this finding my A matrix I wrote down that my A matrix as
A=[pl(:,1).*pr(:,1) pl(:,2).*pr(:,1) pr(:,1) pl(:,1).*pr(:,2) pl(:,2).*pr(:,2) pr(:,2) pl(:,1) pl(:,2), pl(:,3)];
but I'm rather uncertain that this is the right code as I didn't get the same A matrix as a friend of mine. Is it not on the right track? I wanted to do the eight point algorithm without normalization first.

Benefit of applying tf.image.per_image_standardization() over batch_norm layer in Tensorflow?
What is the benefit in applying
tf.image.per_image_standardization()
before the first layer of the deep neural network over adding theBatch_Norm layer
as the first layer?In order to normalize the [0.0, 255.0] float value image pixels before feeding into the network, which method would be suitable?
tf.image.per_image_standardization()
Batch_Norm  layer

Set "training=False" of "tf.layers.batch_normalization" when training will get a better validation result
I use TensorFlow to train DNN. I learned that Batch Normalization is very helpful for DNN , so I used it in DNN.
I use "tf.layers.batch_normalization" and follow the instructions of the API document to build the network: when training, set its parameter "training=True", and when validate, set "training=False". And add tf.get_collection(tf.GraphKeys.UPDATE_OPS).
Here is my code:
# * coding: utf8 * import tensorflow as tf import numpy as np input_node_num=257*7 output_node_num=257 tf_X = tf.placeholder(tf.float32,[None,input_node_num]) tf_Y = tf.placeholder(tf.float32,[None,output_node_num]) dropout_rate=tf.placeholder(tf.float32) flag_training=tf.placeholder(tf.bool) hid_node_num=2048 h1=tf.contrib.layers.fully_connected(tf_X, hid_node_num, activation_fn=None) h1_2=tf.nn.relu(tf.layers.batch_normalization(h1,training=flag_training)) h1_3=tf.nn.dropout(h1_2,dropout_rate) h2=tf.contrib.layers.fully_connected(h1_3, hid_node_num, activation_fn=None) h2_2=tf.nn.relu(tf.layers.batch_normalization(h2,training=flag_training)) h2_3=tf.nn.dropout(h2_2,dropout_rate) h3=tf.contrib.layers.fully_connected(h2_3, hid_node_num, activation_fn=None) h3_2=tf.nn.relu(tf.layers.batch_normalization(h3,training=flag_training)) h3_3=tf.nn.dropout(h3_2,dropout_rate) tf_Y_pre=tf.contrib.layers.fully_connected(h3_3, output_node_num, activation_fn=None) loss=tf.reduce_mean(tf.square(tf_Ytf_Y_pre)) update_ops = tf.get_collection(tf.GraphKeys.UPDATE_OPS) with tf.control_dependencies(update_ops): train_step = tf.train.AdamOptimizer(1e4).minimize(loss) with tf.Session() as sess: sess.run(tf.global_variables_initializer()) for i1 in range(3000*num_batch): train_feature=... # Some processing train_label=... # Some processing sess.run(train_step,feed_dict={tf_X:train_feature,tf_Y:train_label,flag_training:True,dropout_rate:1}) # when train , set "training=True" , when validate ,set "training=False" , get a bad result . However when train , set "training=False" ,when validate ,set "training=False" , get a better result . if((i1+1)%277200==0):# print validate loss every 0.1 epoch validate_feature=... # Some processing validate_label=... # Some processing validate_loss = sess.run(loss,feed_dict={tf_X:validate_feature,tf_Y:validate_label,flag_training:False,dropout_rate:1}) print(validate_loss)
Is there any error in my code ? if my code is right , I think I get a strange result:
when training, I set "training = True", when validate, set "training = False", the result is not good . I print validate loss every 0.1 epoch , the validate loss in 1st to 3st epoch is
0.929624 0.992692 0.814033 0.858562 1.042705 0.665418 0.753507 0.700503 0.508338 0.761886 0.787044 0.817034 0.726586 0.901634 0.633383 0.783920 0.528140 0.847496 0.804937 0.828761 0.802314 0.855557 0.702335 0.764318 0.776465 0.719034 0.678497 0.596230 0.739280 0.970555
However , when I change the code "sess.run(train_step,feed_dict={tf_X:train_feature,tf_Y:train_label,flag_training:True,dropout_rate:1})" , that : set "training=False" when training, set "training=False" when validate . The result is good . The validate loss in 1st epoch is
0.474313 0.391002 0.369357 0.366732 0.383477 0.346027 0.336518 0.368153 0.330749 0.322070 0.335551
Why does this result appear ? Is it necessary to set "training=True" when training, set "training=False" when validate ?

Ignore label(s) when training Fully Convolutional Network
I'm using fully convolutional networks for semantic segmentation in Caffe, using the Cityscapes dataset.
This script allows to convert IDs of classes, and says to set IDs of classes to ignore at 255, and "ignore these labels during training". How do we do that in practice ? I mean, how do I 'tell' my network that 255 is not a true class as the other integers ?
Thanks for giving me an intuition behind it.

Image Classification  Tensorflow
I am building an image classifier for 10 categories using tensorflow. My test accuracy is only around 25%. My parameters are: params = {"input_shape":[200,300,3], "conv_layers": 5, "filters":[40,50,60,60,60], "kernel_size":[5,5], "hidden_units": 9000, "drop_rate":0.4, "n_classes":10} and Relu activation function. I have 1000 training images per category.(300x200) what shall i do to get more accuracy? what alteration can be made in the params?How many epoch will be efficient?

How to handle class imbalance in multi class classification
I am working with multiclass classification problem which has 7 categories in target. There is a huge class imbalance. Is there any built in function in R to handle this scenario? If Yes, could some one please post the function. If no, kindly help me out with some idea of how to handle this.

Categorical classification in Keras Python
I am doing multiclass classification of 5 classes. I am using Tensorflow with Keras. My code is like this:
# load dataset dataframe = pandas.read_csv("Data5Class.csv", header=None) dataset = dataframe.values # split into input (X) and output (Y) variables X = dataset[:,0:47].astype(float) Y = dataset[:,47] print("Load Data.....") encoder= to_categorical(Y) def create_larger(): model = Sequential() print("Create Dense Ip & HL 1 Model ......") model.add(Dense(47, input_dim=47, kernel_initializer='normal', activation='relu')) print("Add Dense HL 2 Model ......") model.add(Dense(40, kernel_initializer='normal', activation='relu')) print("Add Dense output Model ......") model.add(Dense(5, kernel_initializer='normal', activation='sigmoid')) model.compile(loss='categorical_crossentropy', optimizer='adam', metrics=['accuracy']) return model estimators = [] estimators.append(('rnn', KerasClassifier(build_fn=create_larger, epochs=60, batch_size=10, verbose=0))) pipeline = Pipeline(estimators) kfold = StratifiedKFold(n_splits=10, shuffle=True, random_state=seed) results = cross_val_score(pipeline, X, encoder, cv=kfold) print("Accuracy: %.2f%% (%.2f%%)" % (results.mean()*100, results.std()*100))
The CSV file I have taken as an input contains the data with labels. The labels are like this
0, 1, 2, 3, 4
which represent 5 different classes. Then, as the labels are already in integer form, do I need to use
the
LabelEncoder()
function in my code? Also, I have used
to_categorical(Y
) function. Should I use it or I should just pass the Y variable containing these labels to the classifier for training?I got the error like this: Supported target types are: ('binary', 'multiclass'). Got 'multilabelindicator' instead. This error occurred when I used encoder variable in the code results = cross_val_score(pipeline, X, encoder, cv=kfold) where encoder variable represents the to_categorical(Y) data. How to solve this error?
 Then, as the labels are already in integer form, do I need to use
the