Transcript pptx
The Fast Evaluation of Hidden Markov Models on GPU Presented by Ting-Yu Mu & Wooyoung Lee Introduction Hidden Markov Model (HMM): ◦ A statistical method (Probability based) ◦ Used in a wide range of applications: Speech recognition Computer vision Medical image analysis One of the problems need to be solved: ◦ Evaluate the probability of an observation sequence on a given HMM (Evaluation) ◦ The solution of above is the key to choose the best matched models among the HMMs Introduction – Example Application: Speech Recognition Goal: Recognize words one by one Input: ◦ The speech signal of a given word → Represented as a time sequence of coded spectral vectors Output: ◦ The observation sequence → Represented as an index indicator in the spectral codebook Introduction – Example The tasks: ◦ Design individual HMMs for each word of vocabulary ◦ Perform unknown word recognition: Using the solution of evaluation problem to score each HMM based on the observation sequence of the word The model scores the highest is selected as the result The accuracy: ◦ Based on the correctness of the chosen result Computational Load Computational load of previous example consists of two parts: ◦ Estimate the parameters of HMMs to build the models, and the load varies upon each HMM Executed only one time ◦ Evaluate the probability of an observation sequence on each HMM Executed many times on recognition process The performance is depend on the complexity of the evaluation algorithm Efficient Algorithm The lower order of complexity: ◦ Forward-backward algorithm Consists of two passes: Forward probability Backward probability Used extensively Computational intensive ◦ One way to increase the performance: Design the parallel algorithm Utilizing the present day’s multi-core systems General Purpose GPU Why choose Graphic Processing Unit ◦ Rapid increases in the performance Supports floating-points operations Fast computational power/memory bandwidth GPU is specialized for compute-intensive and highly parallel computation More transistors are devoted to data processing rather that data caching CUDA Programming Model The GPU is seen as a compute device to execute part of the application that: ◦ Has to be executed multiple times ◦ Can be isolated as a function ◦ Works independently on different data Such a function can be compiled to run on the device. The resulting program is called a Kernel The batch of threads that executes a kernel is organized as a grid of blocks CUDA Programming Model Thread Block: ◦ Contains the batch of threads that can be cooperate together: Fast shared memory Synchronizable Thread ID ◦ The block can be one-, two-, or threedimensional arrays CUDA Programming Model Grid of Thread Block: ◦ Contains the limited number of threads in a block ◦ Allows larger numbers of thread to execute the same kernel with one invocation ◦ Blocks identifiable through block ID ◦ Leads to a reduction in thread cooperation ◦ Blocks can be one- or two-dimensional arrays CUDA Programming Model CUDA Memory Model Parallel Algorithm on GPU The tasks of computing the evaluation probability is split into pieces and delivered to several threads ◦ A thread block evaluates a Markov model ◦ Calculating the dimension of the grid: Obtained by dividing the number of states N by the block size ◦ Forward probability is computed by a thread within a thread block ◦ Needs to synchronize the threads due to: Shared data CUDAfy.NET What is CUDAfy.Net? Made by Hybrid DSP Systems in Netherlands a set of libraries and tools that permit general purpose programming of NVIDIA CUDA GPUs from the Microsoft .NET framework. combining flexibility, performance and ease of use First release: March 17, 2011 Cudafy.NET SDK Cudafy .NET Library ◦ Cudafy Translator (Convert .NET code to CUDA C) ◦ Cudafy Library (CUDA support for .NET) ◦ Cudafy Host (Host GPU wrapper) ◦ Cudafy Math (FFT + BLAS) The translator converts .NET code into CUDA code. Based on ILSPY (Open Source .NET assembly browser and decompiler) Cudafy Translator GENERAL CUDAFY PROCESS Two main components to the Cudafy SDK: ◦ Translation from .NET to CUDA C and compiling using NVIDIA compiler (this results in a Cudafy module xml file) ◦ Loading Cudafy modules and communicating with GPU from host It is not necessary for the target machine to perform the first step above. ◦ 1. Add reference to Cudafy.NET.dll from your .NET project ◦ 2. Add the Cudafy, Cudafy.Host and Cudafy.Translator namespaces to source files (using in C#) ◦ 3. Add a parameter of GThread type to GPU functions and use it to access thread, block and grid information as well as specialist synchronization and local shared memory features. ◦ 4. Place a Cudafy attribute on the functions. ◦ 5. In your host code before using the GPU functions call Cudafy.Translator.Cudafy( ). This returns a Cudafy Module instance. ◦ 6. Load the module into a GPGPU instance. The GPGPU type allows you to interact seamlessly with the GPU from your .NET code. Development Requirement NVIDIA CUDA Toolkit 4.1 Visual Studio 2010 ◦ Microsoft VC++ Compiler (used by NVIDIA CUDA Compiler) Windows( XP SP3, VISTA, 7 32bit/64bit) NVIDIA GPUs NVIDIA Graphic Driver GPU vs PLINQ vs LINQ GPU vs PLINQ vs LINQ Reference ILSpy : http://wiki.sharpdevelop.net/ilspy.ashx Cudafy.NET : http://cudafy.codeplex.com/ Using Cudafy for GPGPU Programming in .NET : http://www.codeproject.com/Articles/202792/Usi ng-Cudafy-for-GPGPU-Programming-in-NET Base64 Encoding on a GPU : http://www.codeproject.com/Articles/276993/Bas e64-Encoding-on-a-GPU High Performance Queries: GPU vs LINQ vs PLINQ : http://www.codeproject.com/Articles/289551/Hig h-Performance-Queries-GPU-vs-PLINQ-vs-LINQ