This is an old revision of the document!
Table of Contents
Running Auryn simulations on the Intel Xeon Phi (MIC) architecture
The following section is highly experimental. To compile Auryn in native mode for the Intel Xeon Phi cards the following steps worked. Requirements: First, use the newest Auryn development version, which does not compile against GSL any more. This saves you some extra troubel in compiling GSL. Second, you need the Intel Composer suite. Third, you need the boost libraries (see required libraries) compiled for the intel MIC architecture ideally using the Intel MPI implementation which is native to Xeon and the Xeon Phis.
Compiling Boost for Xeon Phi/MIC
For the most part you can follow one of Boosts tutorials for the compilation http://www.boost.org/doc/libs/1_55_0/doc/html/mpi/getting_started.html#mpi.config. In particular I set user-config.jam
to contain the line
using mpi : /opt/intel/impi/4.1.3.048/bin/mpigxx -cxx=/opt/intel/bin/icpc -static_mpi : -L/opt/intel/impi/4.1.3.048/lib ;
to enable MPI and to tell the build toolchain Jam which compiler to use. Note that you might have to adapt the version numbers in this example to your actial version.
Building of boost was then achieved by using a command line similar to the following. It is important to pass the -mmic
arguments to tell the intel compiler that it is cross compiling for the Xeon Phi:
./b2 toolset=intel --disable-icu --without-iostreams cflags="-mmic" cxxflags="-mmic -I/opt/intel/impi/4.1.3.048/mic/include/" linkflags="-mmic" --debug-configuration
Building Boost for Xeon Phi/MIC
With this done one simply has to create a new subdir in the Auryn build directory and add the following Makefile:
.SECONDARY: CC = /opt/intel/impi/4.1.3.048/bin/mpicxx -cxx=/opt/intel/bin/icc CXX = /opt/intel/impi/4.1.3.048/bin/mpicxx -cxx=/opt/intel/bin/icc CFLAGS= -ansi -Wall -pipe -O3 -mmic -static_mpi -static-intel -vec-report2 -pedantic -I$(SRCDIR) -I$(DEVDIR) LDFLAGS= -L/opt/intel/composer_xe_2013_sp1.2.144/compiler/lib/mic/ -lboost_program_options -lboost_serialization -lboost_mpi -limf -lsvml -lirng -lintlc include ../Makefile.include
Again the path variables will probably need to be updated to your needs. Furthermore depending on where you installed the Boost-for-MIC libraries you might have to add a cflag -I/lcncluster/zenke/lib/build/boost_1_55_0/
and similarly a linkflag -L/yourpath/boost_1_55_0/stage/lib
.
Now you are almost set for compilation. However, before you compile icc
generally requires you to update your LD_LIBRARY_PATH
and MIC_LD_LIBRARY_PATH
. This is most conveniently achieved by
source /opt/intel/impi/4.1.3.048/bin/mpivars.sh source /opt/intel/composer_xe_2013_sp1.2.144/bin/compilervars_arch.sh mic
Invoking make
will now build the simulation binaries.
Running Auryn simulations on Xeon Phi/MIC
To run a simulation (e.g. sim_background) in native mode on a Xeon Phi card you need to copy the binary and the required shared libraries to the card. This essentially follows the steps outlined in this Intel tutorial https://software.intel.com/en-us/articles/using-the-intel-mpi-library-on-intel-xeon-phi-coprocessor-systems.
Finally the code can be run on the coprocessor card:
[xyz@srv1-mic0 tmp]# mpirun -n 60 ./sim_background --dir /tmp --simtime 10
Enjoy!