| Next revision | Previous revision |
| computer_science:machine_learning:udacity:intro_to_tensorflow_for_deep_learning:basics_training_your_first_model [2020/08/11 12:32] – created carlossousa | computer_science:machine_learning:udacity:intro_to_tensorflow_for_deep_learning:basics_training_your_first_model [2023/12/01 12:07] (current) – external edit 127.0.0.1 |
|---|
| ====== The Basics: Training your first model ====== | ====== The Basics: Training your first model ====== |
| | |
| | Note: this is part of [[:computer_science:machine_learning:udacity:intro_to_tensorflow_for_deep_learning|Intro to TensorFlow for DeepLearning by Udacity]] |
| | |
| | ---- |
| |
| The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is: | The problem we will solve is to convert from Celsius to Fahrenheit, where the approximate formula is: |
| |
| Next, import [[http://www.numpy.org/|NumPy]] as ''np''. Numpy helps us to represent our data as highly performant lists. | Next, import [[http://www.numpy.org/|NumPy]] as ''np''. Numpy helps us to represent our data as highly performant lists. |
| <code> | <code python> |
| import tensorflow as tf # Import Tensorflow | import tensorflow as tf # Import Tensorflow |
| import numpy as np # Import NumPy | import numpy as np # Import NumPy |
| |
| We create two lists ''celsius_q'' and ''fahrenheit_a'' that we can use to train our model. | We create two lists ''celsius_q'' and ''fahrenheit_a'' that we can use to train our model. |
| <code>celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) | <code python> |
| | celsius_q = np.array([-40, -10, 0, 8, 15, 22, 38], dtype=float) |
| fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) | fahrenheit_a = np.array([-40, 14, 32, 46, 59, 72, 100], dtype=float) |
| |
| </code> | </code> |
| |
| **Some Machine Learning terminology** | ==== Some Machine Learning terminology ==== |
| |
| * **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius. | * **Feature** — The input(s) to our model. In this case, a single value — the degrees in Celsius. |
| * **Labels** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit. | * **Labels** — The output our model predicts. In this case, a single value — the degrees in Fahrenheit. |
| * **Example** — A pair of inputs/outputs used during training. In our case a pair of values from ''celsius_q'' and ''fahrenheit_a'' at a specific index, such as ''(22,72)''. | * **Example** — A pair of inputs/outputs used during training. In our case a pair of values from ''celsius_q'' and ''fahrenheit_a'' at a specific index, such as ''(22,72)''. |
| * ''input_shape=[1]'' — This specifies that the input to this layer is a single value. That is, the shape is a one-dimensional array with one member. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a floating point number, representing degrees Celsius. | * ''input_shape=[1]'' — This specifies that the input to this layer is a single value. That is, the shape is a one-dimensional array with one member. Since this is the first (and only) layer, that input shape is the input shape of the entire model. The single value is a floating point number, representing degrees Celsius. |
| * ''units=1'' — This specifies the number of neurons in the layer. The number of neurons defines how many internal variables the layer has to try to learn how to solve the problem (more later). Since this is the final layer, it is also the size of the model's output — a single float value representing degrees Fahrenheit. (In a multi-layered network, the size and shape of the layer would need to match the ''input_shape'' of the next layer.) | * ''units=1'' — This specifies the number of neurons in the layer. The number of neurons defines how many internal variables the layer has to try to learn how to solve the problem (more later). Since this is the final layer, it is also the size of the model's output — a single float value representing degrees Fahrenheit. (In a multi-layered network, the size and shape of the layer would need to match the ''input_shape'' of the next layer.) |
| <code>l0 = tf.keras.layers.Dense(units=1, input_shape=[1]) | <code python> |
| | l0 = tf.keras.layers.Dense(units=1, input_shape=[1]) |
| model = tf.keras.Sequential([l0]) | model = tf.keras.Sequential([l0]) |
| |
| * **Loss function** — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the "loss".) | * **Loss function** — A way of measuring how far off predictions are from the desired outcome. (The measured difference is called the "loss".) |
| * **Optimizer function** — A way of adjusting internal values in order to reduce the loss. | * **Optimizer function** — A way of adjusting internal values in order to reduce the loss. |
| <code>model.compile(loss='mean_squared_error', | <code python> |
| optimizer=tf.keras.optimizers.Adam(0.1)) | model.compile(loss='mean_squared_error', |
| | optimizer=tf.keras.optimizers.Adam(0.1)) |
| |
| |
| |
| This cycle of calculate, compare, adjust is controlled by the ''fit'' method. The first argument is the inputs, the second argument is the desired outputs. The ''epochs'' argument specifies how many times this cycle should be run, and the ''verbose'' argument controls how much output the method produces. | This cycle of calculate, compare, adjust is controlled by the ''fit'' method. The first argument is the inputs, the second argument is the desired outputs. The ''epochs'' argument specifies how many times this cycle should be run, and the ''verbose'' argument controls how much output the method produces. |
| <code> | |
| | <code python> |
| history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False) | history = model.fit(celsius_q, fahrenheit_a, epochs=500, verbose=False) |
| print("Finished training the model") | print("Finished training the model") |
| |
| We'll use [[https://matplotlib.org/|Matplotlib]] to visualize this (you could use another tool). As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near "perfect" towards the end. | We'll use [[https://matplotlib.org/|Matplotlib]] to visualize this (you could use another tool). As you can see, our model improves very quickly at first, and then has a steady, slow improvement until it is very near "perfect" towards the end. |
| <code> | |
| | <code python> |
| import matplotlib.pyplot as plt | import matplotlib.pyplot as plt |
| plt.xlabel('Epoch Number') | plt.xlabel('Epoch Number') |
| So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? Take a guess before you run this code. | So, for example, if the Celsius value is 100, what do you think the Fahrenheit result will be? Take a guess before you run this code. |
| |
| <code>print(model.predict([100.0])) | <code python> |
| | print(model.predict([100.0])) |
| |
| |
| The correct answer is 100×1.8+32=212, so our model is doing really well. | The correct answer is 100×1.8+32=212, so our model is doing really well. |
| |
| ===== To review ===== | ==== To review ==== |
| |
| * We created a model with a Dense layer | * We created a model with a Dense layer |
| Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, 100 Celsius was not part of our training data.) | Our model tuned the variables (weights) in the Dense layer until it was able to return the correct Fahrenheit value for any Celsius value. (Remember, 100 Celsius was not part of our training data.) |
| |
| ===== Looking at the layer weights ===== | ==== Looking at the layer weights ==== |
| |
| Finally, let's print the internal variables of the Dense layer. | Finally, let's print the internal variables of the Dense layer. |
| |
| <code>print("These are the layer variables: {}".format(l0.get_weights())) | <code python> |
| | print("These are the layer variables: {}".format(l0.get_weights())) |
| |
| |
| |
| Just for fun, what if we created more Dense layers with different units, which therefore also has more variables? | Just for fun, what if we created more Dense layers with different units, which therefore also has more variables? |
| <code> | |
| | <code python> |
| l0 = tf.keras.layers.Dense(units=4, input_shape=[1]) | l0 = tf.keras.layers.Dense(units=4, input_shape=[1]) |
| l1 = tf.keras.layers.Dense(units=4) | l1 = tf.keras.layers.Dense(units=4) |
| The Code: | The Code: |
| |
| <code>import tensorflow as tf # Import Tensorflow | <code python> |
| | import tensorflow as tf # Import Tensorflow |
| import numpy as np # Import NumPy | import numpy as np # Import NumPy |
| import logging | import logging |
| |
| model.compile(loss='mean_squared_error', | model.compile(loss='mean_squared_error', |
| optimizer=tf.keras.optimizers.Adam(0.1)) | optimizer=tf.keras.optimizers.Adam(0.1)) |
| |
| history = model.fit(celsius_q, fahrenheit_a, epochs=750, verbose=False) | history = model.fit(celsius_q, fahrenheit_a, epochs=750, verbose=False) |
| The Output: | The Output: |
| |
| <code>-40.0 degrees Celsius = -40.0 degrees Fahrenheit | <code reg> |
| | -40.0 degrees Celsius = -40.0 degrees Fahrenheit |
| -10.0 degrees Celsius = 14.0 degrees Fahrenheit | -10.0 degrees Celsius = 14.0 degrees Fahrenheit |
| 0.0 degrees Celsius = 32.0 degrees Fahrenheit | 0.0 degrees Celsius = 32.0 degrees Fahrenheit |
| |
| </code> | </code> |
| | |
| | ====== References ====== |
| | |
| | [[https://classroom.udacity.com/courses/ud187|https://classroom.udacity.com/courses/ud187]] |
| | |
| |