ML03: PyTorch vs. TensorFlow

PyTroch—a promising framework for DL

Read time: 3 min
This is a part my mid-term report of the course PyTorch and Machine Learning in NCCU. The original report: https://bit.ly/2UZftXq

Advantages of PyTroch

• PyTorch is more Pyhonic than TensorFlow.
• TensorFlow doesn’t outperform PyTorch on speed.
• Skyrocketingly growing number of PyTorch users.

Disadvantages of PyTroch

• Few resources including books and online discussions about PyTorch compared to those about TensorFlow.

Outline
(1) Architecture of PyTorch & TensorFlow
(2) Performance of PyTorch & TensorFlow
(3) Popularity of PyTorch & TensorFlow
(4) Conclusion
(5) References

(1) Architecture of PyTorch & TensorFlow

Let’s check the architecture (or so-called anatomy) of PyTorch & TensorFlow.

Figure 1: Anatomy of PyTorch [1]

Figure 2: The layered TensorFlow architecture [2]

Figure 2: The layered TensorFlow architecture

The engine and low-level library of PyTorch and TensorFlow are pretty similar and they are basically built by C & C++, so theoretically they shall have similar speed. Then, Let’s shift our attention to the comparisons of PyTorch and TensorFlow in all dimensions.

(2) Performance of PyTorch & TensorFlow

Table 1: Comparisons of Keras, TensorFlow & PyTorch [3]

The green cells in table 1 represent the apparent superiority. Furthermore, since we know the dynamic computation graph of PyTorch would make it more flexible and easier to debug compared with TensorFlow. The dynamic computation graph that PyTorch possess make PyTorch more Pythonic than Tensorflow having static computation graph.

Some of the facts in the figure are quite easy to guess. However, just like Python have more flexibility and debugging capabilities than C/C++, but Python loses in speed. Would PyTorch have slower speed ? Surprisingly, PyTorch’s performances equal to TensorFlow in dimensions like speed and dataset compatibility !

(3) Popularity of PyTorch & TensorFlow

Figure 3: Percent of PyTorch papers of total TensorFlow / PyTorch papers [4]

Figure 4: Percent of framework mentioned by ML papers [5]

Figure 5: Google search results of TensorFlow & PyTorch ((blue, red) = (TensorFlow, PyTorch)) [6]

Additionally, these 3 figures above show how popular PyTorch was lately, and the growing numbers of adopting PyTroch indicating the promising future of PyTroch.

(4) Conclusion

Advantages of PyTroch

• PyTorch is more Pyhonic than TensorFlow.
• TensorFlow doesn’t outperform PyTorch on speed.
• Skyrocketingly growing number of PyTorch users.

Disadvantages of PyTroch

• Few resources including books and online discussions about PyTorch compared to those about TensorFlow.

(5) References

[1] Stevens, E., Antiga, L. & Thomas, V. (2020). Deep Learning with PyTorch. New York, NY: Manning.

[2] Abadi, M. et al. (2016, November). TensorFlow: A System for Large-Scale Machine Learning. Paper presented at the 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI ’16), Savannah, GA. Abstract retrieved from https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf

[3] Rao, C.S.J.(2020). Your First steps in Computer Vision: Using PyTorch with an example. Retrieved from

Using Pytorch with Examples in Computer Vision

[4] He, H.(2020). PyTorch vs TensorFlow. Retrieved from https://chillee.github.io/pytorch-vs-tensorflow/

[5] Migdal, P., & Jakubanis, R. (2018). Keras or PyTorch as your first deep learning framework. Retrieved from

Keras or PyTorch as your first deep learning framework - deepsense.ai

[6] Great Learning Team (2020). PyTorch vs TensorFlow — Explained | What is the difference between PyTorch and TensorFlow?. Retrieved from

Difference between PyTorch and TensorFlow | TensorFlow vs. PyTorch