Research on modern computer architecture optimization techniques: implementation and measurements for big data processing
Date
2021-09-11
Authors
He, Yan
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
With the rapid development of big data computing, our programmers need to improve the efficiency of big data processing. In our daily development process, we normally focus on bug-free deployment and often overlook another aspect of software engineering, performance optimization. Actually, a great deal of programming effort is required to achieve good performance. Many techniques are having the potential to significantly improve software performance. To achieve efficiency, I firstly conclude some optimization techniques dealing with memory-bound issues then moving to parallel programming to deal with compute-bound problems. As an example, we apply as many techniques that I mention in the report to a popular algorithm implementation, PageRank based on C++17. By providing constant feedback from performance measurement and profiling, we can see a drastic speed up after implementing all these optimization techniques. I also experiment with a real-world graph dataset provided by IMDb which can rank the top movies, and also evaluate performance before and after optimization. This report hopefully can provide potential future direction towards applying different optimization techniques to various big data processing applications.
Description
Keywords
big data, data locality, OpenMP, GPU computing, profile data, CUDA, PageRank, movie reference, single-threaded optimization, multi-core optimization