cuBLAS
cuBLAS is NVIDIA's implementation of the BLAS (Basic Linear Algebra Subprograms) interface on the GPU: vector-vector, matrix-vector, and matrix-matrix operations, including the GEMM that Choosing a Library and Programming Tensor Cores both treat as the target hand-written kernels are measured against. The API is small and stable — a handle, a stream, and a handful of call shapes — but it inherits one convention from Fortran BLAS that trips up nearly everyone writing C or C++ against it for the first time.