PETSc: Features: GPU support

# PETSc

# Features: GPU support

PETSc algebraic solvers now run on GPU systems from NVIDIA and AMD. This sometimes provides an alternative high-performance, low-cost solution technique.

We recommend working with petsc master (git branch) if you wish to work witht his feature.

If you use the PETSc GPU code in your published work, please cite the following paper:

            @article{minden2010preliminary,
              title={Preliminary implementation of PETSc using GPUs},
              author={Minden, V. and Smith, B.F. and Knepley, M.G.},
              journal={Proceedings of the 2010 International Workshop of GPU Solutions to Multiscale Problems in Science and Engineering},
              year={2010}
            }

WARNING: Using GPUs effectively is difficult! You must be dedicated and willing to get into the guts of GPU usage if you are serious about using GPUs.

- Installing PETSc to use NVIDIA GPUs (CUDA)
- Installing PETSc to use GPUs independent of the vendor (OpenCL)
- How the GPU solvers are implemented in PETSc
- Example that uses CUSP directly in the user function evaluation
- Presentation on some aspects of GPU usage from PETSc
- Quick summary of usage with CUDA (provided by the CUSP library):
  - The VecType `VECSEQCUSP`, `VECMPICUSP`, or `VECCUSP` may be used with `VecSetType()` or -vec_type seqcusp,mpicusp, or cusp when `VecSetFromOptions()` is used.
  - The MatType `MATSEQAIJCUSP`, `MATMPIAIJCUSP`, or `MATAIJCUSP` maybe used with MatSetType or -mat_type seqaijcusp,mpiaijcusp, or aijcusp when `MatSetOptions()` is used.
  - The MatType `MATSEQAIJCUSPARSE`, `MATMPIAIJCUSPARSE`, or `MATAIJCUSPARSE` maybe used with MatSetType or -mat_type seqaijcusparse,mpiaijcusparse, or aijcusparse when `MatSetOptions()` is used.
  - If you are creating the vectors and matrices with a DM, you can use -dm_vec_type cusp and -dm_mat_type aijcusp
- Quick summary of usage with OpenCL (provided by the ViennaCL library):
  - The VecType `VECSEQVIENNACL`, `VECMPIVIENNACL`, or `VECVIENNACL` may be used with `VecSetType()` or -vec_type seqviennacl,mpiviennacl, or viennacl when `VecSetFromOptions()` is used.
  - The MatType `MATSEQAIJVIENNACL`, `MATMPIAIJVIENNACL`, or `MATAIJVIENNACL` maybe used with MatSetType or -mat_type seqaijviennacl,mpiaijviennacl, or aijviennacl when `MatSetOptions()` is used.
  - If you are creating the vectors and matrices with a DM, you can use -dm_vec_type viennacl and -dm_mat_type aijviennacl
- General hints:
  - It is useful to develop your code with the default vectors and then run production runs with the command line options to use the GPU since debugging on GPUs is difficult.
  - All of the Krylov methods except `KSPIBCGS` run on the GPU. We also provide `PCBICGSTABCUSP`, but it does not appear more efficient than `PCBICGSTAB`.
  - The only preconditioners to run directly on the GPU are `PCJACOBI`, `PCSACUSP`, `PCSACUSPPOLY`, and `PCAINVCUSP`. The `PCBJACOBI` and `PCASM` are just containers, so if the subsolver runs on the GPU, they can also be considered to run on the GPU.
- Some GPU systems (for example many laptops) only run with single precision; thus, PETSc must be built with the `./configure option --with-precision=single`
- We could use your help in further developing PETSc for GPUs; see PETSc Developers site.