Compute Unified Device Architecture --Supercomputing For the MASSES

AUTHOR Havirdhara B.Tech III-II year

AFFILIATIONS: KESHAV MEMORIAL INSTITUTE OF TECHNOLOGY
AFFILIATED TO JNTU
HYDERABAD

E-Mail:- havirdhara@gmail.com

ABSTRACT:

In this age of super-computing, the demand for the extremely high speed processors is surging. This ever-increasing demand forces us to go for the high speed Multi-core processors. Multi-core processors are no longer the future of computing-they are the present day reality. With the rise of multi-core architectures the question of the hour is: how to program massively parallel processors. Nvidia, the pioneer in GPU design, has come up with an advanced user-friendly architecture, “THE CUDA” that enables dramatic increases in computing performance by harnessing the power of the GPU (graphics processing unit). CUDA (an acronym for COMPUTE UNIFED DEVICE ARCHITECTURE) reduces the complexity of the parallel programming to a great extent. The best feature of CUDA is that we can program the GPUs using C, JAVA and other high level programming environments. In this paper, we present the basics of CUDA programming with the need for the evolution of the same. This paper also presents the different applications of CUDA, which tells us why and how CUDA scores over other parallel programming architectures.

Introduction:

Parallelism is the age old technique used for the efficient data processing. The same technique re-emerging into TLP i.e., Thread Level Parallelism and in combination with some finely developed manufacturing technologies provided the world with huge benefits in the form of multi cores. Generally a GPU (Graphical Processing Unit) consists of multiple number of cores for the same reason as gaming is all about “all happening at the same time”, which requires parallelism. Each new generation of GPUs provides flexible programmability and computational power which exceeds previous generations. Nowadays, graphics hardware is capable of executing increasingly complex algorithms, previously handled only with CPUs. Processing non-graphics tasks on GPUs spurred the development of programming models. The GPU (Graphical Processing Unit) is turning into GPGPU (General Purpose Graphical Processing Unit). That means “all at the same time” technique is being applicable to the real time applications other than graphics as well. Various interfaces for high-performance, data-parallel computations exist, among others NVIDIA’s CUDA [NVI07b], AMD’sCTM [PSG06], Brook [BFH * 04] and Sh[MQP02]and their spin-offs Peak Stream and Rapid-Mind. All expose the intrinsic parallelism of GPUs to the user and provide means to perform general-purpose computations. Out of the above interfaces, Nvidia’s CUDA scores over others in various aspects like super scalability and efficiency and design simplicity.

Evolution and Need for CUDA:

Before going into the technical details of CUDA, let’s first discuss the evolution phase of the CUDA. Late 80’s and early 90’s is called “Golden Age of Parallel Processing “.That was the era where a huge interest was created in the field of parallel computing. Fine granularity of data is introduced based on which the then Super Computers were made. Connection machine, MasPar, Cray are some of the machines which were built based on the fine granularity technique. But they were all very expensive and were very hard for the masses to afford them. Then came the era of GPUs. GPUs are massively multithreaded many core chips which consist hundreds of scalar processors. Tens of Thousands of concurrent threads run parallel on the GPUs and have a peak performance of even 1 TFLOP. But this multithreading may be a boon to the users, but from the programming sense, it is quite a complex task. So making the life easy for the parallel-programmers easy is a challenging task, which Nvidia’s dealt with style by introducing CUDA in 2007.

ENTERING THE CUDA:

Now let’s move into the technical details of CUDA. We will be discussing the design goal and the basic details of CUDA now.

Design Goal:

The basic design goal behind CUDA is to make the things easy from the application developers’ point of view. Its objectives are as follows:

• Scalability to 100s of Cores and 1000s of threads

• Let the Programmers focus on parallel algorithms instead of thinking about its implementation

• Enable the se of heterogeneous Systems(CPU and GPU)

• Making the Learning process very easy by only extending C/C++.

So these are the basic ideas involved in CUDA programming. It makes the programmer very happy because he need not think about the lower levels of abstraction. He can focus only on his code and nothing else. And another important thing is that since the CUDA is after all C/C++, anyone can start with CUDA without any knowledge about GPU.

CUDA Program Structure:

A CUDA program consists of one or more phases that are executed on either the host (CPU) or a device such as a GPU. The phases that exhibit little or no data parallelism are implemented in host code. The phases that exhibit rich amount of data parallelism are implemented in the device code. The program supplies a single source code encompassing both host and device code. The NVIDIA C Compiler (NVCC) separates the two. The host code is straight ANSI C code and is compiled with the host's standard C compilers and runs as an ordinary process. The device code is written using ANSI C extended with keywords for labeling data-parallel functions, called kernels, and their associated data structures. The device code is typically further compiled by the NVCC and executed on a GPU device. In situations where there is no device available or the kernel is more appropriately executed on a CPU, one can also choose to execute kernels on a CPU.
The kernel functions, or simply kernels, typically generate a large number of threads to exploit data parallelism.

[pic]

CUDA Execution Model:

The CUDA Execution Model is quite simple. We write an application program in an editor with the necessary syntax (which will actually be very similar to C/C++ language. A sample program is given below), which is divided into a number of parallel and sequential parts. All the parallel parts are made to execute on the separate GPU card instead of the CPU. These parallel programs running on GPU are called kernels.

[pic]

Each Kernel is again executed in the form of many threads as shown in the figure.

[pic]

This actually gives the speed and the fastness to the CUDA programming. But this also comes at a cost of complete independency between the multiple threads. That is each kernel has many threads executing in parallel which are very independent to each other, which means they have less things to share in common, which affects the Thread Cooperation in a big way. To save the application from this problem, CUDA designers came with an idea of grids as shown in the figure.

[pic]

So the threads in a grid have a shared memory and so they can synchronize effectively. This also takes care that the thread synchronized mechanism doesn’t affect the speed much.

Memory Management Model:

The below figure depicts the memory management issue of the CUDA.

[pic]

Per-block, per –thread, Per-device memory sharing mechanism is a very efficient memory-sharing mechanism. The physical memory lay-out gives the outline of the memory management. The data transfer between CPU and GPU takes place through the global memory only. So this is how the CUDA works. The below is the sample cuda code for the matrix operation with suitable explanation. We can infer that the logic and the syntax are very similar to that of C.

SAMPLE CODE:

// incrementArray.cu
#include
#include
#include
void incrementArrayOnHost(float *a, int N)
{ int i; for (i=0; i < N; i++) a[i] = a[i]+1.f; }
__global__ void incrementArrayOnDevice(float *a, int N)
{ int idx = blockIdx.x*blockDim.x + threadIdx.x; if (idx