In the last tutorial, it was said that there are two ways of creating deep learning models in Keras. The first was by using the sequential class where layers of the network are stacked on each other. Although these models are common amongst learners, they, however, have limitations. For instance, with the sequential model, the deep learning architecture cannot be made to share two different layers or have more than one input or output. In real-life applications, this is a bottleneck.
The good news is that the second method; using the functional API, solves this problem. The functional API gives room for more flexibility in the model architecture and is typically ideal for a more complex model. In this tutorial, we shall talk about the Keras functional API and go further to implement it using the same example in the last tutorial where the sequential model was used. By the end of this tutorial you will learn the following:
- Sequential Models
- Using the Keras Functional Models
- Using a Functional Model to Fit a Linear Regression Problem.
- Building a Model with Shared Input Layer
Let’s begin with an overview of the Sequential model.
Sequential Models
Keras sequential model API is used to build deep learning models that allow the different layers to be stacked upon each other sequentially. First, the sequential class must be instantiated then the layers are created and added to it. See an example below
model = Sequential() model.add(Dense(64, input_dim=1, activation='relu')) model.add(Dense(32, input_dim=1, activation='relu')) model.add(Dense(1, activation='linear'))
As seen in the code example above, the sequential class was instantiated with a variable name model. A fully connected layer with 64 nodes was added after which another fully connected layer with 32 nodes was added on top of the first hidden layer. Finally, the output layer with just one node was added. As explained earlier these kinds of models are not perfect for multiple input sources or output.
Using the Keras Functional Models
The Functional Model is another way of creating a deep learning model in Keras. It allows you to create layers that can be reused and have shared inputs and output data. The functional model is typically used for creating a more sophisticated model. Using the Functional Model method can be done in three steps.
Step 1: Define the input
When using the Functional model, you must first define the input layer as a standalone object and pass its shape. This is unlike the case in Sequential models where the shape of the input data is passed alongside the first hidden layer.
The shape of the input data is a tuple of the number of features contained in the data. Note that the tuple must make room for the data splitting (in mini-batch size) that would take place during training. So for data with one feature, the shape is defined as (1, ).
To define the input layer, the Input class must be imported from Keras layers.
from keras.layers import Input #define the input layer input_layer = Input(shape=(1, ))
Step 2: Create and Connect the Layers.
The next step is to create the layers. When creating the layers, you will need to define the number of nodes the layer would have and importantly where the input data would come from. The input data is placed inside a bracket after defining the type of layer you wish to create. See the example below.
from keras.layers import Input #define the hidden layers and the data to be received. hidden_1 = Dense(64)(input_layer)
Now, this is the interesting part. Because you define where the input data comes from for every created layer, you are at liberty to use shared data inputs/outputs for different layers. Thus, making the possibilities with the Functional Models largely sophisticated.
Step 3: Create the model
The final step is to create the model. The Model class has to be first imported from the models module of Keras. The class must then be instantiated, passing the inputs and outputs parameters as variable. The input parameters are the input layer already defined while the output data is the last hidden layer of the model.
from keras.models import Model model = Model(inputs=input_layer, outputs=hidden_1)
The model can then be created and trained on the data. Let’s take an example.
Using a Functional Model to Fit a Linear Regression Problem.
In the last tutorial, we built a Sequential model to be trained on data points and produced the line of best fit. Here, we will do the same with a Function model with Keras. Let’s begin by importing the necessary libraries.
#import the necessary libraries import numpy as np from matplotlib import pyplot as plt from keras.models import Model from keras.layers import Input, Dense, Dropout
Next is to create the dataset.
#define a random seed np.random.seed(42) #create random points for the x and y axis from 0 to 20 xs = np.linspace(0, 20, 50) ys = np.linspace(20, 0, 50) # add some positive and negative noise on the y axis ys += np.random.uniform(-2, 2, 50) #plot the graph plt.plot(xs, ys, '.');
Output:
Next is to build the model. Remember that the Input layer has to be instantiated with the shape of the expected data, Then the other hidden layers will be defined and called on the data it is expected to receive. The model architecture is built in the code below.
#create the input layer input_layer = Input(shape=(1, )) #create the other hidden layers and the output layer hidden_1 = Dense(64, activation='relu')(input_layer) hidden_2 = Dense(32, activation='relu')(hidden_1) output = Dense(1, activation='linear')(hidden_2)
Finally, the Model class is instantiated receiving the already defined input layer and output layer as arguments.
model = Model(inputs=input_layer, outputs=output)
Before going ahead to train the model, we can visualize the model architecture using the plot_model module of keras.
from keras.utils import plot_model plot_model(model)
Output:
As seen, the model has an input layer, two fully connected layers and an output layer. You can get more information about the model architecture by using the summary() method.
model.summary()
Output:
Model: "model"
Total params: 2,241
Trainable params: 2,241
Non-trainable params: 0
The model can finally be compiled and trained.
model.compile(optimizer='adam', loss='mse') history = model.fit(xs, ys, epochs=200)
Output:
Epoch 1/200 2/2 [==============================] - 1s 6ms/step - loss: 122.2916 Epoch 2/200 2/2 [==============================] - 0s 15ms/step - loss: 122.3255 Epoch 3/200 2/2 [==============================] - 0s 5ms/step - loss: 109.8412 Epoch 4/200 2/2 [==============================] - 0s 10ms/step - loss: 113.5361 Epoch 5/200 ... Epoch 196/200 2/2 [==============================] - 0s 10ms/step - loss: 3.6998 Epoch 197/200 2/2 [==============================] - 0s 10ms/step - loss: 4.8764 Epoch 198/200 2/2 [==============================] - 0s 5ms/step - loss: 4.6869 Epoch 199/200 2/2 [==============================] - 0s 5ms/step - loss: 4.5548 Epoch 200/200 2/2 [==============================] - ETA: 0s - loss: 2.816 - 0s 0s/step - loss: 3.6678
The prediction can then be made and the line plotted.
#model makes prediction
y_pred = model.predict(xs)
#plot the graph
plt.plot(xs, y_pred, 'k', label='Predicted')
plt.scatter(xs, ys, label='True')
plt.legend()
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('A plot to show the prediction of the model')
;
Output:
This is a pretty decent result.
Building a Model with Shared Input Layer
The Functional API model particularly shines when you wish to share the data on different layers or have multiple inputs and outputs. If we wish to build a model that shares its input layer, we just need to call the input layer function to different hidden layers. Then you can concatenate it with the concatenate sub-module in Keras. See an example below.
from keras.layers import merge #create the input layer input_layer = Input(shape=(1, )) #create two hidden layers with shared input layer hidden_1 = Dense(64, activation='relu')(input_layer) hidden_2 = Dense(64, activation='relu')(input_layer) #merge the two layers merged_layers = merge.concatenate([hidden_1, hidden_2]) #built the rest of the layers hidden_3 = Dense(32, activation='relu')(merged_layers) output = Dense(1, activation='linear')(hidden_3) model = Model(inputs=input_layer, outputs=output)
Let’s visualize what the model looks like
from keras.utils import plot_model plot_model(model)
Output:
We can also check its summary.
model.summary()
Model: "model"
__________________________________________________________________________________________________
Layer (type) Output Shape Param # Connected to
==================================================================================================
input_1 (InputLayer) [(None, 1)] 0
__________________________________________________________________________________________________
dense (Dense) (None, 64) 128 input_1[0][0]
__________________________________________________________________________________________________
dense_1 (Dense) (None, 64) 128 input_1[0][0]
__________________________________________________________________________________________________
concatenate (Concatenate) (None, 128) 0 dense[0][0]
dense_1[0][0]
__________________________________________________________________________________________________
dense_2 (Dense) (None, 32) 4128 concatenate[0][0]
__________________________________________________________________________________________________
dense_3 (Dense) (None, 1) 33 dense_2[0][0]
==================================================================================================
Total params: 4,417
Trainable params: 4,417
Non-trainable params: 0Running this model also produces good results.
model.compile(optimizer='adam', loss='mse') history = model.fit(xs, ys, epochs=200)
Output:
Epoch 1/200
2/2 [==============================] - 1s 5ms/step - loss: 108.9526
Epoch 2/200
2/2 [==============================] - 0s 5ms/step - loss: 105.3066
Epoch 3/200
2/2 [==============================] - 0s 5ms/step - loss: 99.1887
Epoch 4/200
2/2 [==============================] - 0s 10ms/step - loss: 108.9373
Epoch 5/200
2/2 [==============================] - 0s 5ms/step - loss: 103.2374
...
Epoch 196/200
2/2 [==============================] - 0s 5ms/step - loss: 4.4243
Epoch 197/200
2/2 [==============================] - 0s 5ms/step - loss: 3.9060
Epoch 198/200
2/2 [==============================] - 0s 5ms/step - loss: 4.6090
Epoch 199/200
2/2 [==============================] - 0s 5ms/step - loss: 3.5898
Epoch 200/200
2/2 [==============================] - 0s 0s/step - loss: 3.4363Now, we can plot the predicted values of the model
y_pred = model.predict(xs)
k
plt.plot(xs, y_pred, 'k', label='Predicted')
plt.scatter(xs, ys, label='True')
plt.legend()
plt.xlabel('X Axis')
plt.ylabel('Y Axis')
plt.title('A plot to show the prediction of the model')
;
Output:
In summary,
We have explained how to use this Functional API in Keras. You learned that using this model creates more flexibility in building the architecture of the model and overall its performance. This is why it is mostly used in more complex situations than the sequential model. If you have any questions as to how the model was created, please leave them in the comment section and I’d do my best to answer them.
