    lid velocity = 2.68464e-05, prandtl # = 1., grashof # = 1.
    Number of SNES iterations = 3
    ************************************************************************************************************************
    ***             WIDEN YOUR WINDOW TO 120 CHARACTERS.  Use 'enscript -r -fCourier9' to print this document            ***
    ************************************************************************************************************************

    ---------------------------------------------- PETSc Performance Summary: ----------------------------------------------

    ex19 on a arch-fastmath named Q02-I1-J06.vesta.itd with 16 processors, by balay Thu Aug  1 04:39:11 2013
    Using Petsc Release Version 3.4.2, unknown

                             Max       Max/Min        Avg      Total
    Time (sec):           2.712e+00      1.00000   2.712e+00
    Objects:              7.110e+02      1.00000   7.110e+02
    Flops:                2.286e+08      1.06162   2.192e+08  3.508e+09
    Flops/sec:            8.431e+07      1.06162   8.085e+07  1.294e+09
    MPI Messages:         7.715e+03      1.96510   5.800e+03  9.280e+04
    MPI Message Lengths:  2.919e+06      2.00306   3.822e+02  3.547e+07
    MPI Reductions:       1.828e+03      1.00000

    Flop counting convention: 1 flop = 1 real number operation of type (multiply/divide/add/subtract)
                                e.g., VecAXPY() for real vectors of length N --> 2N flops
                                and VecAXPY() for complex vectors of length N --> 8N flops

    Summary of Stages:   ----- Time ------  ----- Flops -----  --- Messages ---  -- Message Lengths --  -- Reductions --
                            Avg     %Total     Avg     %Total   counts   %Total     Avg         %Total   counts   %Total
     0:      Main Stage: 2.7118e+00 100.0%  3.5079e+09 100.0%  9.280e+04 100.0%  3.822e+02      100.0%  1.827e+03  99.9%

    ------------------------------------------------------------------------------------------------------------------------
    See the 'Profiling' chapter of the users' manual for details on interpreting output.
    Phase summary info:
       Count: number of times phase was executed
       Time and Flops: Max - maximum over all processors
                       Ratio - ratio of maximum to minimum over all processors
       Mess: number of messages sent
       Avg. len: average message length (bytes)
       Reduct: number of global reductions
       Global: entire computation
       Stage: stages of a computation. Set stages with PetscLogStagePush() and PetscLogStagePop().
          %T - percent time in this phase         %f - percent flops in this phase
          %M - percent messages in this phase     %L - percent message lengths in this phase
          %R - percent reductions in this phase
       Total Mflop/s: 10e-6 * (sum of flops over all processors)/(max time over all processors)
    ------------------------------------------------------------------------------------------------------------------------
    Event                Count      Time (sec)     Flops                             --- Global ---  --- Stage ---   Total
                       Max Ratio  Max     Ratio   Max  Ratio  Mess   Avg len Reduct  %T %f %M %L %R  %T %f %M %L %R Mflop/s
    ------------------------------------------------------------------------------------------------------------------------

    --- Event Stage 0: Main Stage

    SNESSolve              1 1.0 2.6176e+00 1.0 2.29e+08 1.1 9.3e+04 3.8e+02 1.8e+03 97100100100 98  97100100100 98  1340
    SNESFunctionEval       4 1.0 2.2562e-03 1.1 8.07e+05 1.0 1.9e+02 9.6e+02 2.0e+00  0  0  0  1  0   0  0  0  1  0  5547
    SNESJacobianEval      21 1.0 7.3912e-01 1.0 1.89e+07 1.1 2.3e+04 2.8e+02 5.9e+02 27  8 25 18 32  27  8 25 18 32   391
    SNESLineSearch         3 1.0 1.0331e-02 1.0 2.04e+06 1.0 2.9e+02 1.3e+03 1.2e+01  0  1  0  1  1   0  1  0  1  1  3065
    VecDot                 3 1.0 5.7602e-04 1.7 5.76e+04 1.0 0.0e+00 0.0e+00 3.0e+00  0  0  0  0  0   0  0  0  0  0  1552
    VecMDot              200 1.0 5.8402e-02 2.9 5.81e+06 1.1 0.0e+00 0.0e+00 2.0e+02  1  3  0  0 11   1  3  0  0 11  1526
    VecNorm              246 1.0 1.9328e-02 1.6 1.46e+06 1.1 0.0e+00 0.0e+00 2.5e+02  1  1  0  0 13   1  1  0  0 13  1160
    VecScale             773 1.0 2.3659e-02 1.1 1.85e+06 1.1 0.0e+00 0.0e+00 0.0e+00  1  1  0  0  0   1  1  0  0  0  1199
    VecCopy              585 1.0 5.4266e-03 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
    VecSet               605 1.0 2.6875e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
    VecAXPY             1563 1.0 4.6154e-02 1.1 6.59e+06 1.1 0.0e+00 0.0e+00 0.0e+00  2  3  0  0  0   2  3  0  0  0  2182
    VecAYPX             1104 1.0 3.5784e-02 1.1 3.00e+06 1.1 0.0e+00 0.0e+00 0.0e+00  1  1  0  0  0   1  1  0  0  0  1283
    VecWAXPY               3 1.0 4.0293e-04 1.1 2.88e+04 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  1109
    VecMAXPY             221 1.0 2.9564e-02 1.1 6.97e+06 1.1 0.0e+00 0.0e+00 0.0e+00  1  3  0  0  0   1  3  0  0  0  3621
    VecPointwiseMult      18 1.0 1.9741e-04 1.1 1.04e+04 1.1 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0   770
    VecScatterBegin     1861 1.0 6.9589e-02 1.4 0.00e+00 0.0 8.8e+04 4.0e+02 0.0e+00  2  0 95 98  0   2  0 95 98  0     0
    VecScatterEnd       1861 1.0 1.4504e-01 2.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  4  0  0  0  0   4  0  0  0  0     0
    VecReduceArith         7 1.0 5.5981e-04 1.0 1.34e+05 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  3726
    VecReduceComm          4 1.0 1.7691e-04 1.6 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00  0  0  0  0  0   0  0  0  0  0     0
    VecNormalize         221 1.0 2.7398e-02 1.3 1.96e+06 1.1 0.0e+00 0.0e+00 2.2e+02  1  1  0  0 12   1  1  0  0 12  1097
    MatMult             1049 1.0 6.2806e-01 1.1 9.45e+07 1.1 5.0e+04 5.3e+02 0.0e+00 23 42 54 76  0  23 42 54 76  0  2318
    MatMultAdd           138 1.0 1.9430e-02 1.2 1.33e+06 1.0 4.6e+03 1.2e+02 0.0e+00  1  1  5  2  0   1  1  5  2  0  1058
    MatMultTranspose     162 1.0 3.5868e-02 1.4 1.56e+06 1.0 5.3e+03 1.2e+02 0.0e+00  1  1  6  2  0   1  1  6  2  0   673
    MatSolve              23 1.0 8.7094e-04 1.2 8.68e+04 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0  1595
    MatSOR              1026 1.0 9.0138e-01 1.1 8.71e+07 1.1 0.0e+00 0.0e+00 0.0e+00 32 38  0  0  0  32 38  0  0  0  1477
    MatLUFactorSym         1 1.0 3.2616e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 3.0e+00  0  0  0  0  0   0  0  0  0  0     0
    MatLUFactorNum         3 1.0 8.9812e-04 1.0 2.73e+04 1.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0   486
    MatAssemblyBegin      37 1.0 1.2632e-02 4.5 0.00e+00 0.0 0.0e+00 0.0e+00 6.8e+01  0  0  0  0  4   0  0  0  0  4     0
    MatAssemblyEnd        37 1.0 2.6602e-02 1.0 0.00e+00 0.0 1.1e+03 7.6e+01 1.0e+02  1  0  1  0  6   1  0  1  0  6     0
    MatGetRowIJ            1 1.0 2.0981e-05 1.2 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
    MatGetOrdering         1 1.0 2.5487e-04 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 4.0e+00  0  0  0  0  0   0  0  0  0  0     0
    MatZeroEntries        21 1.0 2.8384e-03 1.0 0.00e+00 0.0 0.0e+00 0.0e+00 0.0e+00  0  0  0  0  0   0  0  0  0  0     0
    MatFDColorCreate       7 1.0 1.0388e-01 1.0 0.00e+00 0.0 6.7e+02 1.1e+02 5.0e+02  4  0  1  0 27   4  0  1  0 27     0
    MatFDColorApply       21 1.0 6.3318e-01 1.0 1.89e+07 1.1 2.2e+04 2.8e+02 8.7e+01 23  8 24 18  5  23  8 24 18  5   456
    MatFDColorFunc       441 1.0 9.4265e-02 1.4 1.73e+07 1.1 2.1e+04 2.8e+02 1.2e+01  3  8 23 17  1   3  8 23 17  1  2802
    MatGetRedundant        3 1.0 1.6680e-03 1.0 0.00e+00 0.0 1.2e+03 3.8e+02 4.0e+00  0  0  1  1  0   0  0  1  1  0     0
    KSPGMRESOrthog       200 1.0 8.3607e-02 1.8 1.16e+07 1.1 0.0e+00 0.0e+00 2.0e+02  2  5  0  0 11   2  5  0  0 11  2132
    KSPSetUp              33 1.0 1.7611e-02 1.1 0.00e+00 0.0 0.0e+00 0.0e+00 1.1e+02  1  0  0  0  6   1  0  0  0  6     0
    KSPSolve               3 1.0 2.0992e+00 1.0 2.12e+08 1.1 8.9e+04 3.6e+02 1.7e+03 77 93 96 90 92  77 93 96 90 92  1552
    PCSetUp                3 1.0 3.1787e-01 1.0 5.28e+06 1.1 2.4e+04 1.6e+02 1.1e+03 12  2 26 11 58  12  2 26 11 58   244
    PCApply               23 1.0 1.7151e+00 1.0 1.96e+08 1.1 6.4e+04 4.2e+02 5.6e+02 63 86 69 75 31  63 86 69 75 31  1750
    ------------------------------------------------------------------------------------------------------------------------

    Memory usage is given in bytes:

    Object Type          Creations   Destructions     Memory  Descendants' Mem.
    Reports information only for process 0.

    --- Event Stage 0: Main Stage

               Container     1              1          564     0
                    SNES     1              1         1308     0
          SNESLineSearch     1              1          856     0
                  DMSNES     1              1          664     0
                  Vector   286            286      5032112     0
          Vector Scatter    42             42        44184     0
                  Matrix    59             59      3895176     0
      Matrix FD Coloring     7              7      2125028     0
        Distributed Mesh     7              7       103608     0
         Bipartite Graph    14             14        11088     0
               Index Set   236            236       252868     0
       IS L to G Mapping    21             21        67764     0
           Krylov Solver    15             15       209336     0
         DMKSP interface     4              4         2592     0
          Preconditioner    15             15        13312     0
                  Viewer     1              0            0     0
    ========================================================================================================================
    Average time to get PetscTime(): 6.91414e-07
    Average time for MPI_Barrier(): 2.19345e-06
    Average time for zero size MPI_Send(): 1.05649e-05
    #PETSc Option Table entries:
    -da_refine 6
    -log_summary
    -pc_type mg
    #End of PETSc Option Table entries
    Compiled without FORTRAN kernels
    Compiled with full precision matrices (default)
    sizeof(short) 2 sizeof(int) 4 sizeof(long) 8 sizeof(void*) 8 sizeof(PetscScalar) 8 sizeof(PetscInt) 4
    Configure run at: Wed Jul 31 16:13:30 2013
    Configure options: PETSC_ARCH=arch-fastmath --prefix=/gpfs/vesta-fs0/projects/FASTMath/install/petsc/3.4.2.1/linux-rhel_6-ppc64-gcc-4.4.6 --with-cmake=/gpfs/vesta-fs0/projects/FASTMath/install/cmake/2.8.11.1/linux-rhel_6-ppc64-gcc-4.4.6/bin/cmake --with-blas-lib=/gpfs/vesta-fs0/projects/FASTMath/install/blas/19Apr11/linux-rhel_6-ppc64-gcc-4.4.6/lib/libblas.a --with-lapack-lib=/gpfs/vesta-fs0/projects/FASTMath/install/lapack/3.4.2/linux-rhel_6-ppc64-gcc-4.4.6/lib/liblapack.a --with-metis-dir=/gpfs/vesta-fs0/projects/FASTMath/install/metis/5.1.0/linux-rhel_6-ppc64-gcc-4.4.6 --with-parmetis-dir=/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6 --with-hdf5-dir=/gpfs/vesta-fs0/projects/FASTMath/install/hdf5/1.8.11/linux-rhel_6-ppc64-gcc-4.4.6 --with-superlu_dist-include=/gpfs/vesta-fs0/projects/FASTMath/install/superlu/dist-3.3/linux-rhel_6-ppc64-gcc-4.4.6/include --with-superlu_dist-lib="-L/gpfs/vesta-fs0/projects/FASTMath/install/superlu/dist-3.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lsuperlu" --with-zoltan-dir=/gpfs/vesta-fs0/projects/FASTMath/install/zoltan/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6 --with-moab-include=/gpfs/vesta-fs0/projects/FASTMath/install/moab/4.6.0/linux-rhel_6-ppc64-gcc-4.4.6/include --with-moab-lib="-L/gpfs/vesta-fs0/projects/FASTMath/install/moab/4.6.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lMOAB -L/gpfs/vesta-fs0/projects/FASTMath/install/cgma/12.2.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lcgm -L/gpfs/vesta-fs0/projects/FASTMath/install/zoltan/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/lib -lzoltan -L/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lparmetis -lmetis -L/gpfs/vesta-fs0/projects/FASTMath/install/hdf5/1.8.11/linux-rhel_6-ppc64-gcc-4.4.6/lib -lhdf5" --with-ml-include=/gpfs/vesta-fs0/projects/FASTMath/install/ml/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/include --with-ml-lib="-L/gpfs/vesta-fs0/projects/FASTMath/install/ml/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/lib -lml -L/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lparmetis -lmetis" --with-hypre-dir=/gpfs/vesta-fs0/projects/FASTMath/install/hypre/2.9.2a/linux-rhel_6-ppc64-gcc-4.4.6 --with-netcdf-dir=/gpfs/vesta-fs0/projects/FASTMath/install/netcdf/4.3.0/linux-rhel_6-ppc64-gcc-4.4.6 --download-chaco --download-exodusii --with-shared-libraries=0 --with-debugging=0 COPTFLAGS="-g -O2" FOPTFLAGS="-g -O2" CXXOPTFLAGS="-g -O2" --with-etags=0
    -----------------------------------------
    Libraries compiled on Wed Jul 31 16:13:30 2013 on vestalac1
    Machine characteristics: Linux-2.6.32-358.6.2.el6.ppc64-ppc64-with-redhat-6.3-Santiago
    Using PETSc directory: /home/balay/petsc
    Using PETSc arch: arch-fastmath
    -----------------------------------------

    Using C compiler: mpicc  -Wall -Wwrite-strings -Wno-strict-aliasing -Wno-unknown-pragmas -g -O2  ${COPTFLAGS} ${CFLAGS}
    Using Fortran compiler: mpif90  -Wall -Wno-unused-variable -g -O2   ${FOPTFLAGS} ${FFLAGS}
    -----------------------------------------

    Using include paths: -I/home/balay/petsc/arch-fastmath/include -I/home/balay/petsc/include -I/home/balay/petsc/include -I/home/balay/petsc/arch-fastmath/include -I/gpfs/vesta-fs0/projects/FASTMath/install/superlu/dist-3.3/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/hypre/2.9.2a/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/ml/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/zoltan/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/metis/5.1.0/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/moab/4.6.0/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/netcdf/4.3.0/linux-rhel_6-ppc64-gcc-4.4.6/include -I/gpfs/vesta-fs0/projects/FASTMath/install/hdf5/1.8.11/linux-rhel_6-ppc64-gcc-4.4.6/include -I/bgsys/drivers/V1R2M0/ppc64 -I/bgsys/drivers/V1R2M0/ppc64/comm/sys/include -I/bgsys/drivers/V1R2M0/ppc64/spi/include -I/bgsys/drivers/V1R2M0/ppc64/spi/include/kernel/cnk -I/bgsys/drivers/V1R2M0/ppc64/comm/gcc/include
    -----------------------------------------

    Using C linker: mpicc
    Using Fortran linker: mpif90
    Using libraries: -Wl,-rpath,/home/balay/petsc/arch-fastmath/lib -L/home/balay/petsc/arch-fastmath/lib -lpetsc -L/gpfs/vesta-fs0/projects/FASTMath/install/superlu/dist-3.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lsuperlu -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/hypre/2.9.2a/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/hypre/2.9.2a/linux-rhel_6-ppc64-gcc-4.4.6/lib -lHYPRE -L/bgsys/drivers/V1R2M0/ppc64/spi/lib -L/bgsys/drivers/V1R2M0/ppc64/comm/sys/lib -L/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc/powerpc64-bgq-linux/4.4.6 -L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/lib/gcc -L/bgsys/drivers/toolchain/V1R2M0-efix23/gnu-linux/powerpc64-bgq-linux/lib -Wl,-rpath,/bgsys/drivers/V1R2M0/ppc64/comm/gcc/lib -lcxxmpich -L/gpfs/vesta-fs0/projects/FASTMath/install/ml/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/lib -lml -L/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lparmetis -lmetis -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/lapack/3.4.2/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/lapack/3.4.2/linux-rhel_6-ppc64-gcc-4.4.6/lib -llapack -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/blas/19Apr11/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/blas/19Apr11/linux-rhel_6-ppc64-gcc-4.4.6/lib -lblas -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/zoltan/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/zoltan/trilinos-11.2.4/linux-rhel_6-ppc64-gcc-4.4.6/lib -lzoltan -Wl,-rpath,/home/balay/petsc/arch-fastmath/lib -L/home/balay/petsc/arch-fastmath/lib -lchaco -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/parmetis/4.0.3/linux-rhel_6-ppc64-gcc-4.4.6/lib -lparmetis -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/metis/5.1.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/metis/5.1.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lmetis -lpthread -lexoIIv2for -lexodus -L/gpfs/vesta-fs0/projects/FASTMath/install/moab/4.6.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lMOAB -L/gpfs/vesta-fs0/projects/FASTMath/install/cgma/12.2.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lcgm -lzoltan -lparmetis -lmetis -L/gpfs/vesta-fs0/projects/FASTMath/install/hdf5/1.8.11/linux-rhel_6-ppc64-gcc-4.4.6/lib -lhdf5 -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/netcdf/4.3.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -L/gpfs/vesta-fs0/projects/FASTMath/install/netcdf/4.3.0/linux-rhel_6-ppc64-gcc-4.4.6/lib -lnetcdf -Wl,-rpath,/gpfs/vesta-fs0/projects/FASTMath/install/hdf5/1.8.11/linux-rhel_6-ppc64-gcc-4.4.6/lib -lhdf5_hl -lhdf5 -lm -lmpichf90 -lgfortran -lm -lm -lcxxmpich -ldl -lmpich -lopa -lmpl -lpami -lSPI -lSPI_cnk -lrt -lpthread -lstdc++ -lgcc_eh -lnss_files -lnss_dns -lresolv -ldl
    -----------------------------------------