System Name Vikram-100
Site Name Physical Research Lab (PRL)
Site Description The Physical Research Laboratory (PRL) is a National Research Institute for space and allied sciences, supported mainly by Department of Space, Government of India. This research laboratory has ongoing research programmes in astronomy and astrophysics, atmospheric sciences and aeronomy, Earth sciences, Solar System studies and theoretical physics. It manages the Udaipur Solar Observatory and is located in Ahmedabad.
Department Computer Science
City Ahmedabad
State Gujarat

Vendor Lenovo/IBM (OEM), CMC Limited (SI)
Installation Date May 5, 2015


SYSTEM

Description IBM/Lenovo NeXtScale System-M5 based Servers, Xeon E5-2670v3 12C 2.3 GHz, Nvidia K40 GPU, Infiniband FDR Interconnect with 300TB IBM GPFS parallel file system. The system having 2328 CPU Cores/194 CPU Socket/77 Pure CPU Compute Node and 11520 GPU Cuda Cores/40 GPU/20 Hybrid(CPU+GPU)Compute Nodes. Total memory of 25.5 TB.
Has two master nodes each with 2X Intel Xeon E5-2670 v3 (48 Cores) and 2 X Nvidia K40 GPU.
Network Dual 4X FDR Infiniband primary interconnect, and dual Gigabit Ethernet secondary interconnect
Node 2x Intel Xeon Processor E5-2670 v3 twelve-core processor at 2.355 GHz, 262 GB RAM, 30 MB L1 cache, 2 x 900 GB SAS 10k RPM, with RedHat Enterprise Linux.
Twenty nodes contain 2 X Nvidia K40 GPU.
Power 68 MWatts Rated Power Unit for the entire system
Cooling 5 Units of InRow Cooling RP DX with each 29 kw Cooling capacity
Footprint 528 sq. ft.


BENCHMARK

Description Two runs of HPL2.1  were made - CPU only and CPU-GPU nodes
MPI Intel impi 5.0.3.048, Open MPI
Compiler Intel composer_xe_2015.2.164, GCC 4.4.7
 
CPU Only:  
Nodes and Cores 77 nodes, 1848 cores
Nmax 870912
 
CPU-GPU:  
Nodes and Cores 20 nodes containing 40 CPUs, 480 CPU cores, and 40 GPUs, 115200 GPU cores
Nmax 558080