This reference material is intended for users who want to use the computational resources of their NVIDIA GPU board for BLAS. Users who do not have the NVIDIA GPU board can ignore this section.

NVIDIA Corp. implemented certain Level 1, 2 and 3 BLAS in their Library, CUDA CUBLAS Library, V3.1, July, 2010. The NVIDIA external names and argument protocols are different from the equivalent Fortran names and argument addressing. See Table 9.2 for names marked in the color GREEN. IMSL has written these marked Fortran BLAS so that they call equivalent NVIDIA C language codes from the CUBLAS library. No direct use or knowledge of C is required by a Fortran programmer in order to take advantage of these codes. It is necessary that a user code or application package be compiled with a Fortran 2003 compiler that has implemented the C Interoperability Standard feature. See The Fortran 2003 Handbook, Adams, et al., p. 561. IMSL’s use of this feature is the key to providing a portable version of these Fortran-callable IMSL/NVIDIA BLAS. The program or application is then compiled and linked using IMSL and NVIDIA libraries that contain these BLAS.

The strategy for using the attached NVIDIA GPU is given by the following algorithm:

Normally a code that calls a IMSL/NVIDIA BLAS code does not have to be aware of the copy steps or the switchover size, NSTART. These are hidden from the user code. In the first algorithm step, a working block is allocated on the GPU for each array argument. A table within the IMSL module, CUBLAS_LIBRARY, records the sizes and GPU addresses of these blocks. If the sizes are too small for the current problem size and data type the blocks are reallocated to be of adequate size. The same working block on the GPU may be used for many calls to the IMSL/NVIDIA BLAS. The IMSL versions of the BLAS also allow a user to define individual values of NSTART for each routine. This is important because using the GPU may be slower than using a CPU Fortran version until a switchover array size is reached. Thereafter the GPU version is typically faster and increasingly much faster as the problem size increases. The default value of NSTART=32 is used for each vector/matrix argument of each routine but it may not be optimal. This default allows the routines to function correctly without initial attention to this value.

The user can change the default switchover value for all IMSL/NVIDIA BLAS vector/matrix arguments by setting NSTART to the desired value prior to calling the BLAS routine. Additionally, users can reset this value for each individual vector/matrix argument of the routines listed in Table 9.2 and marked with the color GREEN by using the IMSL routine CUBLAS_SET(…). Note that CUBLAS_SET cannot be used prior to an initial call to a BLAS code. The switchover values can be obtained using the IMSL routine CUBLAS_GET(…).

The floating point results obtained using the CPU vs. the GPU will likely differ in units of the low order bits in each component. These differences come from non-equivalent strategies of floating point arithmetic and rounding modes that are implemented in the NVIDIA board. This can be an important detail when comparing results for purposes of benchmarking or code regression. Generally either result should be acceptable for numerical work.

As an added feature, the user can flag when the data values for a vector or matrix are present on the GPU and hence suppress the IMSL/NVIDIA BLAS code from first copying the data. This is often important since the data movement from the CPU to the GPU may be a significant part of the computation time. If there is no indication that the data is present, it is copied from the CPU to the GPU each time a routine is called. The necessity of copying for each use of a BLAS code depends on the application. Valid results are always copied back from the GPU to the CPU memory. The indication that data for that positional array argument requires no initial copy step is that the switchover value for that array argument is negative. The absolute value is used as the switchover value. Caution: Be sure and reset this to a positive value when the argument requires an initial copy step.

In Table 9.3 through Table 9.5, we list an enumeration that includes the routines in Table 9.2 marked with the color GREEN. Note the prefix to each name joined with the string ‘CUDABLAS_’. There are enumerated names that currently do not use the NVIDIA hardware. They are included in anticipation of future additions that will use the CUBLAS library.

There are four utility routines provided in the IMSL module CUDABLAS_LIBRARY that can be used to help manage the use of NVIDIA BLAS. These utilities appear in Table 9.7 and are described in more detail in the routines section of these notes.

For example, to set the value at 500 wherein the GPU is first used for the Level-2 routine ‘SGEMV’ first positional array argument, ‘A(*,*)’, i.e. Array_Arg = 1, execute the code:

When the positional array argument, ‘A(*,*)’ does not have to be copied for each subsequent use of ‘SGEMV’:

Some NVIDIA hardware does not have working double precision versions of BLAS because there is no double precision arithmetic available. However, the double precision code itself is part of the CUDA CUBLAS library. It will appear to execute even though it will not give correct results when the device has no double precision arithmetic. When the IMSL software detects that the correct results are not returned, a warning error message will be printed. The user may instruct the application to henceforth use the Fortran code by setting the switchover value to zero. For example, if it is known that the hardware does not support DOUBLE PRECISION, then a code that has calls to ‘DGEMM’ will use an alternate version of this routine. Therefore, ignoring the error message and continuing the code will result in using the alternate version to compute the result. That code would include:

If it is necessary to know if the GPU or the CPU version of ‘SGEMM’ was used following a call to that code, the inquiry code would include:

Table 9.8 lists a number of NVIDIA Helper subprograms called within the CUDABLAS_LIBRARY Modules. These are mostly for internal use only but are documented in the case that a knowledgeable NVIDIA Library user chooses to make use of them.

In Table 9.8 the arguments c_ptr, x, y, A, devA, and B are C pointers to arrays either on the GPU or the CPU. These are instantiated with calls to helper routine cublasAlloc() or by use of the Fortran 2003 intrinsic function c_loc(…) for array arguments residing on the CPU. This intrinsic returns a C pointer to a Fortran object. The helper function cublasError()is called from each of the double precision IMSL/NVIDIA BLAS codes to assess the availability of double precision floating point hardware on the GPU.

The NVIDIA Environmental Subprograms listed in Table 9.9 provide details about the runtime working environment.

One argument for cudaGetDeviceProperties is a Fortran derived type, cudaDeviceProp, with a C binding. This contains technical information about the device, including its name. This C character string, NAME(*), is terminated with C_NULL_CHAR. The derived type, cudaDeviceProp is described below:

© 2005–2014 by NVIDIA Corporation. All rights reserved.

Portions of the NVIDIA SGEMM and DGEMM library routines were written by Vasily Volkov and are subject to the Modified Berkeley Software Distribution License as follows:

Copyright (©) 2007-09, Regents of the University of California

All rights reserved.

Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. ( See CUDA CUBLAS Library,Version 3.1, July, 2010, for these remaining conditions.)