v 1.5.0-1

[sgsokol]

Hi Brian,

Getting familiar with github took some time but here is my proposal for commit.
While cheking by automatic R tool I've got the following output:
[root@gba-03264 R]# R CMD check --as-cran --install-args="--configure-args='--with-lapack=/usr/local/atlas/lib64/libsatlas.so --with-magma-home=/usr/local/src/magma-1.5.0 --with-r-home=/usr/local/src/R-3.1.1'" magma_1.5.0-1.tar.gz

• using log directory ‘/home/sokol/dev/R/magma.Rcheck’
• using R version 3.1.1 (2014-07-10)
• using platform: x86_64-unknown-linux-gnu (64-bit)
• using session charset: UTF-8
• checking for file ‘magma/DESCRIPTION’ ... OK
• this is package ‘magma’ version ‘1.5.0-1’
• checking CRAN incoming feasibility ... NOTE
  Maintainer: ‘Brian J. Smith brian-j-smith@uiowa.edu’
  New maintainer:
  Brian J. Smith brian-j-smith@uiowa.edu
  Old maintainer(s):
  Brian J Smith brian-j-smith@uiowa.edu
  CRAN repository db overrides:
  OS_type: unix
• checking package namespace information ... OK
• checking package dependencies ... NOTE
  No repository set, so cyclic dependency check skipped
• checking if this is a source package ... OK
• checking if there is a namespace ... OK
• checking for executable files ... OK
• checking for hidden files and directories ... OK
• checking for portable file names ... OK
• checking for sufficient/correct file permissions ... OK
• checking whether package ‘magma’ can be installed ... OK
• checking installed package size ... OK
• checking package directory ... OK
• checking DESCRIPTION meta-information ...It is recommended to use ‘given’ instead of ‘middle’.
  It is recommended to use ‘given’ instead of ‘middle’.
  OK
• checking top-level files ... OK
• checking for left-over files ... OK
• checking index information ... OK
• checking package subdirectories ... OK
• checking R files for non-ASCII characters ... OK
• checking R files for syntax errors ... OK
• checking whether the package can be loaded ... OK
• checking whether the package can be loaded with stated dependencies ... OK
• checking whether the package can be unloaded cleanly ... OK
• checking whether the namespace can be loaded with stated dependencies ... OK
• checking whether the namespace can be unloaded cleanly ... OK
• checking loading without being on the library search path ... OK
• checking dependencies in R code ... NOTE
  le package 'methods' est utilisé mais pas déclaré
  See the information on DESCRIPTION files in the chapter ‘Creating R
  packages’ of the ‘Writing R Extensions’ manual.
• checking S3 generic/method consistency ... OK
• checking replacement functions ... OK
• checking foreign function calls ... OK
• checking R code for possible problems ... OK
• checking Rd files ... OK
• checking Rd metadata ... OK
• checking Rd line widths ... OK
• checking Rd cross-references ... OK
• checking for missing documentation entries ... OK
• checking for code/documentation mismatches ... OK
• checking Rd \usage sections ... OK
• checking Rd contents ... OK
• checking for unstated dependencies in examples ... OK
• checking line endings in C/C++/Fortran sources/headers ... OK
• checking line endings in Makefiles ... OK
• checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
• checking compiled code ... NOTE
  Note: information on .o files is not available
  File ‘/home/sokol/dev/R/magma.Rcheck/magma/libs/magma.so’:
  Found ‘exit’, possibly from ‘exit’ (C)

Compiled code should not call entry points which might terminate R nor
write to stdout/stderr instead of to the console. The detected symbols
are linked into the code but might come from libraries and not actually
be called.

See ‘Writing portable packages’ in the ‘Writing R Extensions’ manual.

• checking examples ... OK
• checking PDF version of manual ... WARNING
  LaTeX errors when creating PDF version.
  This typically indicates Rd problems.
  LaTeX errors found:
  !pdfTeX error: pdflatex (file ec-inconsolata): Font ec-inconsolata at 540 not f
  ound
  ==> Fatal error occurred, no output PDF file produced!
• checking PDF version of manual without hyperrefs or index ... OK

WARNING: There was 1 warning.
NOTE: There were 4 notes.
See
‘/home/sokol/dev/R/magma.Rcheck/00check.log’
for details.

Please review and commit if everything is OK.

Best,
Serguei.

[brian-j-smith]

@sgsokol - Great! Thanks for taking time to figure out github. I went ahead and merged your commits into a develop branch. I will try my best to get this latest version tested on our Ubuntu system this week (work permitting) and keep you posted. I'm really looking forward to having the package up and running again.

[sgsokol]

@brian-j-smith Hi,
Had you time to test the patch on your Ubuntu? If so, I suppose that your silence means that things didn't go as smooth as we would like to ;)
Looking forward to have some news from tou.

[brian-j-smith]

I did. Unfortunately, no news isn't good news in this case, although the news is not all bad. We had to spend a little time upgrading our system to the latest version of CUDA (6.5) and magma (1.5.0). Those seem to be up and running okay. I was then able to get your patch compiled and installed in R (3.1.1), but then ran into some bumps :-(. Some of the operators and functions are working, but some are giving errors... then work came calling before I could figure out exactly what was causing the problems.

I have some time this weekend (finally) to have another go at getting things working. I'll check back in on Monday, hopefully with some more specific details.

Thanks for being patient.

[sgsokol]

Good. Feel free to post error messages if you want some help on problem diagnosis.
Keep going ;)

[sgsokol]

Brian,

I have got some erros on my side too.
A colleague of mine gave me a root access on a Linux machine with CentOS 7.0.1406 and GPU card GeForce 8400 GS Rev. 2 (cuda capability 1.1). I could install cuda-6.5 and magma 1.5.0 (where I had to replace arch=10 by arch=11 as the support for 10 has been removed from cuda).

Installation of R magma package 1.5.0-1 went smoothly but the tests failed at matrix multiplication.
I have checked the dgemm test in magma library (magma-1.5.0/testing/testing_dgemm) and it failed too. I am trying to downgrade to cuda-6.0 to see if the bug is there too.

I'll keep you informed.

[sgsokol]

On this card magma-1.5.0 does not work proprely with cuda-6.0 neither :(

[brian-j-smith]

I'm glad you tried that out. Very helpful. I guess the good news is that it wasn't just me. Our card is 1.3 compatible, and I had made a similar change to the arch option to get things working. I did notice messages in recent versions of CUDA warning that 1.x compatibility is going to be dropped soon and that those cards may produce incorrect results. At least your result gives me a good idea of where to start looking this weekend.

[sgsokol]

More news on tracking cublas issue.
I have asked myself till what arch the problem with cublas dgemm are persistent?
Unfortunatly, on a gpu car with capability 3.0 there are still problems. Only the 3.5 is doing dgemm well. Here two examples from magma-lib testing dir. Dev=0 corresponds to cap. 3.5 and dev=1 to 3.0:
$ ./testing_dgemm -l --dev 0
MAGMA 1.5.0 compiled for CUDA capability >= 1.0
CUDA runtime 6000, driver 6000. OpenMP threads 8.
device 0: Tesla K20c, 705.5 MHz clock, 4799.6 MB memory, capability 3.5
device 1: Quadro K2000, 954.0 MHz clock, 2047.7 MB memory, capability 3.0
Usage: ./testing_dgemm [options] [-h|--help]

If running lapack (option --lapack), MAGMA and CUBLAS error are both computed
relative to CPU BLAS result. Else, MAGMA error is computed relative to CUBLAS result.

transA = No transpose, transB = No transpose

M N K MAGMA Gflop/s (ms) CUBLAS Gflop/s (ms) CPU Gflop/s (ms) MAGMA error CUBLAS error

1088 1088 1088 503.98 ( 5.11) 709.42 ( 3.63) 17.48 ( 147.35) 4.34e-15 4.34e-15 failed
2112 2112 2112 577.72 ( 32.61) 1012.82 ( 18.60) 18.33 (1027.96) 5.58e-15 5.58e-15 failed
3136 3136 3136 578.43 ( 106.64) 1013.24 ( 60.88) 18.50 (3333.69) 7.75e-15 7.75e-15 failed
4160 4160 4160 576.33 ( 249.83) 1027.26 ( 140.16) 18.01 (7994.28) 8.79e-15 8.96e-15 failed
5184 5184 5184 568.08 ( 490.47) 1034.00 ( 269.47) 18.13 (15371.76) 1.03e-14 1.03e-14 failed

(even if the test is labeled as failed the cublas error is arounf 10^-14 that's ok for me)
$ ./testing_dgemm -l --dev 1
MAGMA 1.5.0 compiled for CUDA capability >= 1.0
CUDA runtime 6000, driver 6000. OpenMP threads 8.
device 0: Tesla K20c, 705.5 MHz clock, 4799.6 MB memory, capability 3.5
device 1: Quadro K2000, 954.0 MHz clock, 2047.7 MB memory, capability 3.0
Usage: ./testing_dgemm [options] [-h|--help]

If running lapack (option --lapack), MAGMA and CUBLAS error are both computed
relative to CPU BLAS result. Else, MAGMA error is computed relative to CUBLAS result.

transA = No transpose, transB = No transpose

M N K MAGMA Gflop/s (ms) CUBLAS Gflop/s (ms) CPU Gflop/s (ms) MAGMA error CUBLAS error

1088 1088 1088 27.56 ( 93.46) 6205.51 ( 0.42) 17.55 ( 146.80) 4.34e-15 9.98e-01 failed
2112 2112 2112 27.85 ( 676.65) 427169.18 ( 0.04) 18.36 (1025.97) 5.58e-15 9.99e-01 failed
3136 3136 3136 27.90 (2211.10) 1340481.28 ( 0.05) 18.50 (3333.48) 7.75e-15 9.99e-01 failed
4160 4160 4160 27.92 (5156.62) 3212270.01 ( 0.04) 18.03 (7983.62) 8.79e-15 1.00e+00 failed

(here the cublas error is around 10⁰ => too bad. Note that magma errors are low and acceptbale)

[brian-j-smith]

Well this is kind of frustrating and a bit of a head-scratcher. I don't understand why the 3.0 compatible card would have such large errors while the 3.5 card (albeit a much more sophisticated one) seems to be okay.

I haven't had much luck in getting things to work on our 1.3 card, but would be willing to concede it is not worth spending too much time on that given the 1.x cards will be not be supported much longer.

If you get to a point where you have the package working on 2.x and 3.x cards, I would be happy to turn the maintenance reigns over to you. I pushed some configuration changes up to develop branch and, if you like, can say a few words about those as well as my experience getting the package installed in different Linux environments.

[sgsokol]

You are right, it's sad to see a 3.0 card can produce such errors. I'll continue to try to get things working at least on 3.0 (for now, I don't have an access to 2.x card). If I manage to do so, it's ok for me be a contributor to the package. You can continue to be maintainer if you are willing to do it.
I've got a look on the dev brach and tried it on my machine, both on 3.0 and 3.5 cards. It worked "out of the box", all tests are OK (remember that magma errors in dgemm tests were small contrary to pure cublas call).
I would be very interseted to hear about your experience in porting the package to different Linux environments.

[brian-j-smith]

Sounds good. I'll work on getting a 2.0 card to have something up and running for development on my end. You're welcome to keep contributing and I'll maintain - it's not as glamorous as it looks :)

As for different Linux environments... I thought your configure file looked pretty good. The one thing I did noticed was a R_INCLUDE_DIR="${R_HOME}/include" line that looked to be added manually. I had to remove that to get the install to work on our system. There are any number of locations that R include files might be located. If R is being run from its source file directory, then include files can be found in ${R_HOME}/include. However, if R is installed (make install), then they may be located elsewhere. Consequently, R sets the environment variable R_INCLUDE_DIR to the path of its include files when it is installed. I was assuming that users would have R installed and was relying on R_INCLUDE_DIR being set automatically by R, instead of setting it to a value in the configure file - which might have seemed odd.

At least that is my understanding of how R_INCLUDE_DIR works, but I should ask whether that is your experience. In other words, does a configure file that does not manually set the R include path work for you, if R is installed?

[sgsokol]

I don't have much experience in R on multiple Linux distributions. For example, on a station with 1.1 gpu card that I used for previous tests, R is installed via system package mangement tool. Its binary is /usr/bin/R, its R_HOME is defined as /usr/lib64/R and includes go to /usr/include/R. I suppose that you are talking about a case like this. Well, it turns out that in this particular case the things worked well even with R_INCLUDE_DIR set to ${R_HOME}/include as the installer has put a symbolic link in this place pointing to /usr/include/R.
To make configure script more robust, we can borrow a tric that I've seen in another configure
(http://svn.rforge.net/org/trunk/rosuda/JRI/configure.ac):
# find R_INCLUDE_DIR
: ${R_INCLUDE_DIR="${RBIN}" CMD sh -c 'echo $R_INCLUDE_DIR'}
if test -z "${R_INCLUDE_DIR}"; then
echo "could not determine R_INCLUDE_DIR"
exit 1
fi
which onviously requires an RBIN var but a simple which R should work for this everywhere.

[sgsokol]

Brian,

I have left a comment on magma formum about my problem with 3.0 card :
http://icl.cs.utk.edu/magma/forum/viewtopic.php?f=2&t=1056&p=3109#p3109
and it turns out that it was a tester problem, not magma itself. After applying a
patch the testing_dgemm works as expected. I'll ask also for 1.1 card too.

[sgsokol]

Well, for 1.1 card I have found the answer by myself. By isolating a call to cublaDgemm(), I've got a return status CUBLAS_STATUS_ARCH_MISMATCH, i.e. this card does not support double precision.

[brian-j-smith]

Oh yeah, I like the include trick. Our system doesn't have the include symbolic link in the R_HOME (/usr/lib64/R) directory. Maybe a non-standard install was done by our IT group. They don't let me have admin privileges, so I have to live with what they do :(

That makes sense about the 1.1 card. It's been so long, I had forgotten about it not supporting double precision.