Improving Capsule Networks using zero-skipping and pruning
| dc.contributor.author | Sharifi, Ramin | |
| dc.contributor.supervisor | Baniasadi, Amirali | |
| dc.contributor.supervisor | Gulliver, T. Aaron | |
| dc.date.accessioned | 2021-11-15T23:57:37Z | |
| dc.date.available | 2021-11-15T23:57:37Z | |
| dc.date.copyright | 2021 | en_US |
| dc.date.issued | 2021-11-15 | |
| dc.degree.department | Department of Electrical and Computer Engineering | |
| dc.degree.level | Master of Applied Science M.A.Sc. | en_US |
| dc.description.abstract | Capsule Networks are the next generation of image classifiers. Although they have several advantages over conventional Convolutional Neural Networks (CNNs), they remain computationally heavy. Since inference on Capsule Networks is timeconsuming, thier usage becomes limited to tasks in which latency is not essential. Approximation methods in Deep Learning help networks lose redundant parameters to increase speed and lower energy consumption. In the first part of this work, we go through an algorithm called zero-skipping. More than 50% of trained CNNs consist of zeros or values small enough to be considered zero. Since multiplication by zero is a trivial operation, the zero-skipping algorithm can play a massive role in speed increase throughout the network. We investigate the eligibility of Capsule Networks for this algorithm on two different datasets. Our results suggest that Capsule Networks contain enough zeros in their Primary Capsules to benefit from this algorithm. In the second part of this thesis, we investigate pruning as one of the most popular Neural Network approximation methods. Pruning is the act of finding and removing neurons which have low or no impact on the output. We run experiments on four different datasets. Pruning Capsule Networks results in the loss of redundant Primary Capsules. The results show a significant increase in speed with a minimal drop in accuracy. We also, discuss how dataset complexity affects the pruning strategy. | en_US |
| dc.description.scholarlevel | Graduate | en_US |
| dc.identifier.bibliographicCitation | Ramin Sharifi, Pouya Shiri, and Amirali Baniasadi. Zero-skipping in capsnet. is it worth it? In International Conference on Computers and Their Applications, volume 69, pages 355–361, 2020. | en_US |
| dc.identifier.uri | http://hdl.handle.net/1828/13501 | |
| dc.language | English | eng |
| dc.language.iso | en | en_US |
| dc.rights | Available to the World Wide Web | en_US |
| dc.subject | Capsule Network | en_US |
| dc.subject | CapsNet | en_US |
| dc.subject | Deep Learning | en_US |
| dc.title | Improving Capsule Networks using zero-skipping and pruning | en_US |
| dc.type | Thesis | en_US |