Wavenet (using Tensorflow) showing '2 steps together' after restarting previous training
Only having trained once before, my output was about 1,115 sec/step and "files length: 1" was displayed after every step.
I stopped the training, and restarted it using the "-restore_from=/logdir/train/..." option, the output is now a bit different.
It almost seems like it is taking 2 steps together somehow, but I don't understand enough about what could potentially be happening.
Should I adjust something, or is everything fine even though the steps look so different?
See also questions close to this topic
Why is my tf.keras.Model not updated after I restore the checkpoint?
Below is the example code that I am running to understand the behavior of tfe.Checkpoint.
import tensorflow as tf tf.enable_eager_execution() import tensorflow.contrib.eager as tfe import os # Create inputs = tf.keras.Input(shape=(2,)) outputs = tf.keras.layers.Dense(3, activation=tf.nn.relu, use_bias=False)(inputs) model = tf.keras.Model(inputs=inputs, outputs=outputs) print(model.variables) # Modify print('Modify model:') model.variables.assign([[1., 2.,3],[1.,2.,3]]) print(model.variables) # Save this_checkpoint = tfe.Checkpoint(model=model) checkpoint_directory = './tmp3' checkpoint_prefix = os.path.join(checkpoint_directory, "ckpt") save_path = this_checkpoint.save(checkpoint_prefix) print('Saved model as %s' %(save_path)) # Restore print('\nNew model:') inputs2 = tf.keras.Input(shape=(2,)) outputs2 = tf.keras.layers.Dense(3, activation=tf.nn.relu, use_bias=False)(inputs2) model2 = tf.keras.Model(inputs=inputs2, outputs=outputs2) print(model2.variables) ckpt_to_restore = tf.train.latest_checkpoint(checkpoint_directory) print('\nRestoring model from %s' %(ckpt_to_restore)) new_checkpoint = tfe.Checkpoint(model=model2) new_checkpoint.restore(ckpt_to_restore) print(model2.variables)
The output is:
[<tf.Variable 'dense/kernel:0' shape=(2, 3) dtype=float32, numpy= array([[ 0.72318137, -1.05397141, -0.79438519], [-0.64990056, 0.88330412, 1.07289195]], dtype=float32)>] Modify model: [<tf.Variable 'dense/kernel:0' shape=(2, 3) dtype=float32, numpy= array([[ 1., 2., 3.],[ 1., 2., 3.]], dtype=float32)>] Saved model as ./tmp3/ckpt-1 New model: [<tf.Variable 'dense_1/kernel:0' shape=(2, 3) dtype=float32, numpy= array([[-0.74109459, 0.93528509, -0.05304849], [-0.87230837, 0.74049175, -0.26022822]], dtype=float32)>] Restoring model from ./tmp3/ckpt-1 [<tf.Variable 'dense_1/kernel:0' shape=(2, 3) dtype=float32, numpy= array([[-0.74109459, 0.93528509, -0.05304849], [-0.87230837, 0.74049175, -0.26022822]], dtype=float32)>]
I was expecting the dense layer in model2 to get the values [[1,2,3],[1,2,3]]. Why is the layer not updated to the values saved in the checkpoint?
Installing tensorflow with gpu support nvcuda.dll error
I am trying to install tensorflow with gpu support.
I followed this instruction
Yet i recieve error:
Could not find 'nvcuda.dll'. Tensorflow requires that this DL be installed in directory that is named in your %PATH% env. variable
I have add variable as its stated in the guide.
WHat could be the problem? THanks for help.
Keras LSTM go_backwards usage
I have a question regarding the usage of the go_backwards argument in the Keras LSTM model layer. The documentation for this layer can be found here: https://keras.io/layers/recurrent/#lstm.
Question1: If I set the "go_backwards" flag to True, do I still feed the input data "forwards" during the training process. For example, if an input sentence in English normally reads "I fell", and it's German translation reads "Ich fiel", would I feed it forwards ("I fell", "Ich fiel"), or backwards ("fell I", "fiel Ich") during the training process.
Question 2: Same question for making model predictions, is the data fed forward ("I fell"), or reverse ("fell I")?
I think I should feed all data forward but I can't find any documentation that convinces me that this is correct.
How to get similar meaning of phrase using nlp in python?
I need to get similar meaning from phrases from SNOW(Service now) Incident ticket like.. Assume that there is a ticket description like:
"status ticket" -> ticket status "ticket status" -> ticket status "status of ticket" -> ticket status "password reset" -> password issues "reset password" -> password issues "forget password" -> password issues
I need to consider like status ticket = ticket status = status of ticket and vice versa. likewise "password reset" = "reset password" = "forget password" = "password issue" and vice versa. Is it possible to map those keywords to form a similar meaning phrase.. Thanks in advance
AlexNet . Doubts about fully connected layer
I'm trying to understand AlexNet architecture. Activation volume is 7x7x512... We need 3 fully connected layer (that is so because we use 3 max pooling layer according to info of https://es.mathworks.com/help/nnet/ref/nnet.cnn.layer.fullyconnectedlayer.html) with output of 4096 , 4096 and 1000 filters.
I know the filters of the last fully connected layer have to be the same number of classes that we want to classify. My doubt is about the 4096 filters. Usually, this kind of nets used to convert FC to ConvNet, that means we apply a convolutional layer with the same size of filters (7x7 in this case) and the same number of filters to get a vector 1x1xM (4096 in this case). where does that number (4096) come from?
I loocked at this post but i couldn't find the article mentioned "(x) see section 3.2 of the article: the fully-connected layers are first converted to convolutional layers " : How to calculate the number of parameters of convolutional neural networks?
Thanks and feel free to tell my if my arguments are wrong!!
How does Google Translate provide offline translations?
The thing I am specifically puzzled by is how each language corresponds to a single translation file. This means that there must be a common intermediate translation that connects the different languages to solve the NxM problem. How exactly does it work?