Comparison of tensorflow, Keras and pytorch

Comparison of tensorflow, Keras and pytorch

There are many frameworks for deep learning. For example, the following table shows the number of frameworks used in machine learning papers submitted to arXiv.

This shows that tensorflow is currently the most used, followed by Keras and PyTorch. tensorflow is the most popular framework in the conference because it was developed by google and has been used for a long time. tensorflow is a very useful framework because it provides a variety of scalable products, deployment options, and a variety of levels of model abstraction (i.e., whether to build a model by simply calling the name of a layer or to modify a deeper algorithm).

Keras is a high-level API that works with Tensorflow, CNTK, Theano, etc. as a backend, has a simple structure, and is highly readable, a feature that can save time when looking up documentation during development.

PyTorch, on the other hand, is a low-level API that directly manipulates arrays, and is becoming the preferred solution for academic research and deep learning applications that require optimization, and has been receiving a lot of attention lately.

To compare the readability in each framework, an example of a deep learning network description is shown below.

First, the description in tensorflow is as follows.

def conv2d(x, W, b, strides=1):
    x = tf.nn.conv2d(x, W, strides=[1, strides, strides, 1], padding='SAME')
    x = tf.nn.bias_add(x, b)
    return tf.nn.relu(x)
 
def maxpool2d(x, k=2):
    return tf.nn.max_pool(x, ksize=[1, k, k, 1], strides=[1, k, k, 1],padding='SAME')
 
 
def conv_net(x, weights, biases):  
    conv1 = conv2d(x, weights[0], biases[0])
    conv1 = maxpool2d(conv1, k=2)
    conv2 = conv2d(conv1, weights[1], biases[1])
    conv2 = maxpool2d(conv2, k=2)
    fc1 = tf.reshape(conv2, [-1, weights[2].get_shape().as_list()[0]])
    fc1 = tf.add(tf.matmul(fc1, weights[2]), biases[2])
    fc1 = tf.nn.relu(fc1)
    out = tf.add(tf.matmul(fc1, weights[3]), biases[3])
    return out
 
 
pred = conv_net(x, weights, biases)
 
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=pred, labels=y))

Next, we show the description of Keras, which can be expressed in a very simple form compared to tensrflow.

model = Sequential()
model.add(Conv2D(32, (3, 3), activation='relu', input_shape=(32, 32, 3)))
model.add(MaxPool2D())
model.add(Conv2D(16, (3, 3), activation='relu'))
model.add(MaxPool2D())
model.add(Flatten())
model.add(Dense(10, activation='softmax')) 

Finally, we show the description of pythorch, which is a bit more complex than keras.

class Net(nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, 3)
        self.conv2 = nn.Conv2d(32, 16, 3)
        self.fc1 = nn.Linear(16 * 6 * 6, 10)
        self.pool = nn.MaxPool2d(2, 2)
    def forward(self, x):
        x = self.pool(F.relu(self.conv1(x)))
        x = self.pool(F.relu(self.conv2(x)))
        x = x.view(-1, 16 * 6 * 6)
        x = F.log_softmax(self.fc1(x), dim=-1)
        return x
model = Net()

The data comparing the processing speed of each framework is shown below.

It can be seen that the speed of tensorflow and pytorch are almost the same, but the speed of Kera is slower.

In summary, it is most efficient to use Keras for hypothesis testing with small-scale data in the early stages of a task because it is easy to build and highly readable (almost no debugging is required). is the best approach.

These implementations are described in the section on setting up a python development environment on a mac and introducing the tensflow package. Keras is discussed in the Deep Learning with Python and Keras section below, and pytorch is discussed in the Evolving Deep Learning with PyTorch section.