Showing posts with label cpmd. Show all posts
Showing posts with label cpmd. Show all posts

13 September 2012

235. CPMD with Netlib's lapack, blas and your own fftw3 on ROCKS 5.4.3/CentOS 5.6

Update 8 Feb 2013:
I somehow had forgot to include some of the instructions for the BLAS part. Fixed now.

Post:
This is done pretty much like how it's done on Debian (-march=native didn't work in the BLAS compilation though, nor was -fno-whole-file accepted when compiling cpmd)

1. Compile cmake according to this post:
http://verahill.blogspot.com.au/2012/05/compiling-openbabel-231-and-cmake-on.html

2. Compile BLAS
sudo mkdir /share/apps/tools/netlib/blas/lib -p
sudo chown $USER /share/apps/tools/netlib -R
mkdir ~/tmp
cd ~/tmp
wget http://www.netlib.org/blas/blas.tgz
tar xvf blas.tgz
cd BLAS/

Edit make.inc
OPTS     = -O3 -shared -m64 -fPIC

make all

gfortran -shared -Wl,-soname,libnetblas.so -o libblas.so.1.0.1 *.o -lc
ln -s libblas.so.1.0.1 libnetblas.so
cp lib*blas* /share/apps/tools/netlib/blas/lib

3. Compile LAPACK
sudo mkdir /share/apps/tools/netlib/lapack -p
sudo chown $USER /share/apps/tools/netlib -R
wget http://www.netlib.org/lapack/lapack-3.4.1.tgz

tar xvf lapack-3.4.1.tgz
cd lapack-3.4.1/
mkdir build
cd build
ccmake ../

Hit 'c' and edit the values:

 BUILD_COMPLEX                    ON
 BUILD_COMPLEX16                  ON
 BUILD_DOUBLE                     ON
 BUILD_SHARED_LIBS                ON
 BUILD_SINGLE                     ON
 BUILD_STATIC_LIBS                ON
 BUILD_TESTING                    ON
 CMAKE_BUILD_TYPE                    
 CMAKE_INSTALL_PREFIX             /share/apps/tools/netlib/lapack
 LAPACKE                          OFF
 LAPACKE_WITH_TMG                 OFF
 USE_OPTIMIZED_BLAS               ON
 USE_XBLAS                        OFF

Hit 'c' again, then hit 'g'.

Edit CMakeCache.txt and add the following lines at the beginning:

########################
# EXTERNAL cache entries
########################
BLAS_FOUND:STRING=TRUE
BLAS_GENERIC_FOUND:BOOL=TRUE
BLAS_GENERIC_blas_LIBRARY:FILEPATH=/share/apps/tools/netlib/blas/lib/libnetblas.so
BLAS_LIBRARIES:PATH=/share/apps/tools/netlib/blas/lib/libnetblas.so

Do
ccmake ../
again, hit 'c', then 'g'.

Now,
make
make install

4. Compile FFTW3
sudo mkdir /share/apps/tools/fftw3
sudo chown $USER /share/apps/tools/fftw3
cd ~/tmp
wget http://www.fftw.org/fftw-3.3.1.tar.gz
tar -xvf fftw-3.3.1.tar.gz
cd fftw-3.3.1
make distclean
./configure --enable-float --enable-mpi --enable-threads --with-pic --prefix=/share/apps/tools/fftw3/single
make
make install
make distclean
./configure --disable-float --enable-mpi --enable-threads --with-pic --prefix=/share/apps/tools/fftw3/double
make 
make install

5. Compile CPMD
I downloaded the cpmd file to a client computer, then uploaded it to the ROCKS front node:
sftp me@rocks:/home/me/tmp

Connected to rocks.
Changing to: /home/me/tmp
sftp> put cpmd-v3_15_3.tar.gz
Uploading cpmd-v3_15_3.tar.gz to /home/me/tmp/cpmd-v3_15_3.tar.gz
cpmd-v3_15_3.tar.gz                100% 2937KB 587.4KB/s   00:05
sftp> exit

I then logged in via ssh as normal.
cd ~/tmp
tar xvf cpmd-v3_15_3.tar.gz
cd CPMD/CONFIGURE
Create a new file LINUX-x86_64-ROCKS

     IRAT=2
     CFLAGS='-c -O2 -Wall'
     CPP='/lib/cpp -P -C -traditional'
     CPPFLAGS='-D__Linux -D__PGI -D__GNU -DFFT_FFTW3 -DPARALLEL -DPOINTER8'
     FFLAGS='-c -O2 -fcray-pointer -fsecond-underscore'
LFLAGS='-L/share/apps/tools/fftw3/double/lib -lfftw3-lfftw3_mpi -lfftw3_threads -I/usr/include -L/share/apps/tools/netlib/blas/lib -lnetblas -L/share/apps/tools/netlib/lapack/lib -llapack -L/opt/openmpi/lib -lpthread -lmpi'
     FFLAGS_GROMOS='  $(FFLAGS)' 
      FC='mpif77 -fbounds-check'
      CC='mpicc'
      LD='mpif77 -fbounds-check'

NOTE: I don't think the -I belongs in the LFLAGS statement, but I'm presuming that I put it there for a reason back when I did it the first time.

Go to ~/tmp/CPMD, and edit wfnio.F (basically replace 3 with 2 and remove 'L'):

 15       CHARACTER(len=*) TAG
 63         IF(TAG(1:2).EQ.'NI') THEN
201       IF(TAG(1:2).NE.'NI') THEN
271         IF(TAG(1:2).EQ.'NI') THEN

Finally, edit Makefile and change
  23 LD = f95 -O
to
  23 LD = mpif77 -fbounds-check

Time to compile


./mkconfig.sh LINUX-x86_64-ROCKS > Makefile
make
sudo mkdir /share/apps/cpmd
sudo chown $USER /share/apps/cpmd
cp cpmd.x /share/apps/cpmd

echo 'export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/share/apps/tools/netlib/blas/lib:/share/apps/tools/netlib/lapack/lib:/share/apps/tools/fftw3/double/lib' >>~/.bashrc
echo 'export PATH=$PATH:/share/apps/cpmd' >> ~/.bashrc
echo "export PP_LIBRARY_PATH=/share/apps/cpmd/PP_LIBRARY" >>~/.bashrc

You're now done compiling. To test, you need to get some pseudopotential files -- look at e.g. the end of http://verahill.blogspot.com.au/2012/07/not-solved-compiling-cpmd-on-debian.html for instructions.


234. CPMD with netlib lapack, blas and your own fftw on debian testing

This is a minor update to my previous post on CPMD. Back in the days I had issue linking to my Openblas libs (got a binary which would not run properly) but I've since had success with the netlib lapack and blas libs.

1. Compile the netlib lapack and blas libraries according to this post: http://verahill.blogspot.com.au/2012/09/compiling-netlibs-lapack-and-blas-on.html

2. Compile the fftw libraries according to this post (ignore the sections on Openblas and Gromacs):
http://verahill.blogspot.com.au/2012/05/gromacs-with-external-fftw3-and-blas-on.html

3. Compile CPMD. We'll be following this post in large parts.
Register with cpmd.org. Once you're approved download the cpmd source to ~/tmp.

sudo apt-get install libopenmpi-dev openmpi-bin

cd ~/tmp
tar -xvf cpmd-v3_15_3.tar.gz
cd CPMD/CONFIGURE
Create the file LINUX-x86_64-DEBIAN:
   
     IRAT=2
     CFLAGS='-c -O2 -Wall'
     CPP='/lib/cpp -P -C -traditional'
     CPPFLAGS='-D__Linux -D__PGI -D__GNU -DFFT_FFTW3 -DPARALLEL -DPOINTER8'
     FFLAGS='-c -O2 -fcray-pointer -fno-whole-file -fsecond-underscore'
     LFLAGS='-l:/opt/fftw/fftw-3.3.2/double/lib/libfftw3.a -l:/opt/fftw/fftw-3.3.2/double/lib/libfftw3_mpi.a -l:/opt/fftw/fftw-3.3.2/double/lib/libfftw3_threads.a -I/usr/include -l:/opt/netlib/blas/lib/libnetblas.so -l:/opt/netlib/lapack/lib/liblapack.so -lpthread -lmpi'
     FFLAGS_GROMOS='  $(FFLAGS)' 
      FC='mpif77 -fbounds-check'
      CC='mpicc'
      LD='mpif77 -fbounds-check'

Next edit ~/tmp/CPMD/wfnio.F and change the following lines:
 15       CHARACTER(len=*) TAG
 63         IF(TAG(1:2).EQ.'NI') THEN
201       IF(TAG(1:2).NE.'NI') THEN
271         IF(TAG(1:2).EQ.'NI') THEN

Now, in ~/tmp/CPMD, run
./mkconfig.sh LINUX-x86_64-DEBIAN > Makefile
make
sudo mkdir /opt/cpmd
sudo chown $USER /opt/cpmd
cp cpmd.x /opt/cpmd


And follow everything below 'Done! Almost.' in this post: http://verahill.blogspot.com.au/2012/07/not-solved-compiling-cpmd-on-debian.html


24 May 2012

162. PSPW/Carr-Parrinello using ECCE

This is more of a note to self about carr-parrinello using ecce/nwchem. As always, this isn't about the science, but about making the computation run at all. And what I may consider a bug may in fact be a feature.

If you simply click your way through ecce and try to launch a pspw carr-parrinello calc, it will fail.

Two problems:

  • task pspw carr-parrinello expects a .movecs file to be present. You can 'solve' this by putting task pspw steepest_descent before you task pspw carr-parrinello statement
  • if you relaunch a run, often you get a crash with an error referring to writing after EOF. You can solve this by cleaning out your run directory.

Problem number one comes down to this:


"Velocity Wavefunction Datafile
The one-electron orbital velocities are stored in a velocity wavefunction datafile. This is a binary file and cannot be directly edited. This datafile is used by the Car-Parrinello task and can be generated using the v_wavefunction_initializer task."


The dependence on certain files (at a minimum, .movecs) being present and the need for optimisation before CPMD and the fact that either
task pspw energy
or
task pspw optimize
will more often than not prevent your ecce.out file from showing any of the MD stuff means it's still better to set up your file by hand and run everything by hand in a dedicated directory. Ecce doesn't copy runtime file back and forth, and that's the main problem here.

Really, what I find a problem is that I'd like to optimize and equilibriate a set of molecules, then continue using the equilibriated set.

If all you're trying to do is to get something, anything to work to get a feel for how this stuff works, then continue reading.

Problematic example file:

  1 scratch_dir /home/me/jobs/scratch
  2 Title "biphenyl_ground_twisted_cpmd_1-1"
  3
  4 Start  biphenyl_ground_twisted_cpmd_1-1
  5
  6 echo
  7
  8 charge 0
  9
 10 geometry autosym units angstrom
 11  C     0.00676622     3.53807     0.0197363
 12  C     -1.29633     2.88855     0.554869
 13  C     -1.31879     1.38415     0.519460
 14  C     0.00129627     0.730174     -0.000557722
 15  C     1.28746     1.38368     -0.578129
 16  C     1.31931     2.90453     -0.512952
 17  C     -0.0100394     -0.758319     -0.0224661
 18  C     1.33004     -1.36336     0.563945
 19  C     1.24425     -2.89848     0.485842
 20  C     -1.31683     -1.36948     -0.531559
 21  C     0.0254501     -3.54181     0.0318405
 22  C     -1.30632     -2.89413     -0.540694
 23  H     0.0916004     4.71976     0.273191
 24  H     -2.74374     3.75562     1.47791
 25  H     -2.78549     0.594633     1.40665
 26  H     2.70470     3.74589     -1.20122
 27  H     3.09496     -3.77313     1.52949
 28  H     -2.76973     -0.640827     -1.49262
 29  H     0.0203915     -4.70472     -0.288098
 30  H     -2.76848     -3.72979     -1.38695
 31  H     2.88815     -0.631319     1.39550
 32  H     2.66933     0.621436     -1.58686
 33 end
 34
 35 ecce_print ecce.out
 36
 37 nwpw
 38   mult 1
 39   np_dimensions -1  -1
 40   tolerances 1e-7  1e-7
 41   car-parrinello
 42     time_step 5.000000e+00
 43     fake_mass 5.000000e+02
 44     loop 10 100
 45     scaling 1.000000e+00 1.000000e+00
 46   end
 47 end
 48
 49 task pspw car-parrinello

Quick 'solution'
3 memory 200mw
48 task pspw steepest_descent

The line numbers are added by me. Remove them before running.

You can also stick task pspw energy or optimize in there -- but the way ecce does it, with just a task pspw carr-parrinello, won't work. Either way, it'd be nice to be able to carry over the movecs files between calculations.


See below for various errors:


Error #1:
If you set up the run using ecce, it won't work and there won't be any real error message to explain why the run exits immediately.

294      >>>  JOB STARTED       AT Thu May 24 14:19:04 2012  <<<
295           ================ input data ========================
296   library name resolved from: compiled reference
297   NWCHEM_NWPW_LIBRARY set to: </opt/nwchem/nwchem-6.1/src/nwpw/libraryps/>
298   library name resolved from: compiled reference
299   NWCHEM_NWPW_LIBRARY set to: </opt/nwchem/nwchem-6.1/src/nwpw/libraryps/>
300
301 -----ECCE Log Information-----
302 Starting Job: Thu May 24 14:19:02 EST 2012
303 Using /home/me/jobs/scratch as nwchem SCRATCH_DIR
304 nwchem exit status = -1
305 Final exit status = -1
306 Completed Job: Thu May 24 14:19:05 EST 2012

If you launch the run in the terminal (without mpirun -- mpi suppresses error messages sometimes) you get:
     >>>  JOB STARTED       AT Thu May 24 14:20:15 2012  <<<
          ================ input data ========================
  library name resolved from: compiled reference
  NWCHEM_NWPW_LIBRARY set to: </opt/nwchem/nwchem-6.1/src/nwpw/libraryps/>
  library name resolved from: compiled reference
  NWCHEM_NWPW_LIBRARY set to: </opt/nwchem/nwchem-6.1/src/nwpw/libraryps/>
ERROR:  Could not open pipe from input file
The reason is that ecce doesn't carry files over from previous simulations -- you need the .movecs file. This can be generated by
  task pspw steepest_descent 

If you could run all your job in the same directory that wouldn't be a problem.

Error #2
438      >>>  JOB STARTED       AT Thu May 24 14:24:38 2012  <<<
439           ================ input data ========================
440  ------------------------------------------------------------------------
441  out of heap memory        0
442  ------------------------------------------------------------------------
443  ------------------------------------------------------------------------
444   current input line :
445     48: task pspw energy
446  ------------------------------------------------------------------------
447  ------------------------------------------------------------------------
448  ------------------------------------------------------------------------
449  For more information see the NWChem manual at http://www.nwchem-sw.org/        index.php/NWChem_Documentation

We chucked task pspw steepest_descent in before our task pspw carr-parrinello and now get a new error: out of heap memory. Easily fixed. You can set e.g. 200MW under pspw/details or add
memory 200 MW
by hand.

Of course, if you add it by clicking in ecce then your task pspw steepest_descent line will be removed, so you'll have to add that by hand again.


Error #3
According to the manual "This [movecs] datafile is used by the Car-Parrinello task and can be generated using the v_wavefunction_initializer task."
Well, try
task v_wavefunction_initialize
and you get

>>>> PSPW Serial Module - v_wavefunction_initializer <<<<
0:Segmentation Violation error, status=: 11
(rank:0 hostname:beryllium pid:24675):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/signaltrap.c:SigSegvHandler():310 cond:0
Last System Error Message from Task 0:: No such file or directory
And you find that there's a file called ????


Error 4
Going full out:

task pspw wavefunction_initializer
task pspw pseudopotential_formatter
task pspw v_wavefunction_initializer
task pspw car-parrinello
gives


 >>>> PSPW Serial Module - wavefunction_initializer <<<<
0:Floating Point Exception error, status=: 8
(rank:0 hostname:beryllium pid:26026):ARMCI DASSERT fail. ../../ga-5-1/armci/src/common/signaltrap.c:SigFpeHandler():249 cond:0
Last System Error Message from Task 0:: No such file or directory

Error 5:
If you haven't cleared out your run directory you get this via ecce
        ============ Car-Parrinello iteration ==============
     >>>  ITERATION STARTED AT Thu May 24 18:03:07 2012  <<<
    iter.         KE+Energy             Energy        KE_psi        KE_ion   Temperature
    ------------------------------------------------------------------------------------
      10  -0.1662131203E+02  -0.1662582428E+02   0.43690E-02   0.39005E-02        143.80

-----ECCE Log Information-----
Starting Job: Thu May 24 18:00:41 EST 2012

and this if you run in the terminal

At line 847 of file cpmdv5.F (unit = 31, file = './cpmd_test.emotion')
Fortran runtime error: Sequential READ or WRITE not allowed after EOF marker, possibly use REWIND or BACKSPACE
      10   0.1175850345E+06   0.3940717762E+03   0.18793E-03   0.27950E-02       1852.03
--------------------------------------------------------------------------
mpirun has exited due to process rank 0 with PID 26303 on
node beryllium exiting without calling "finalize". This may
have caused other processes in the application to be
terminated by signals sent by mpirun (as reported here).
------------------------------------------------------------------------