Large-Scale Density Functional Calculations

James E. Raynolds, College of Nanoscale Science and Engineering Lenore R. Mullin, College of Computing and Information

Overview

• Using computers to carry out “

numerical experiments

” in Materials Science, Chemistry and Physics • Quantum Mechanical equations solved for a system of atoms in a representative unit cell • Measurable properties obtained from

“first-principles”

– mechanical, thermodynamic, electronic – optical, magnetic, transport

Example: Transport in molecular wire

+ V Phenolate/Benzenediazonium Benzene

Peierls Distortion

dimerized pair Pi stacked pair mechanical relaxation

metal insulator

Frontier Problems

• Non-equilibrium spin-transport in metals and semiconductors (Spintronics) • Transport and coupled mechanical / electronic interactions in molecules (metal - insulator transition due to mechanical relaxation) • Industrial applications: Modeling Chemical Vapor Deposition (CVD) processes atom by atom • Challenges: correlated motion of electrons • Coupled electron-phonon interactions

(electron - vibration coupling)

Density Functional Theory

• Density Functional Theory (DFT) is a “mean-field” solution to the many-electron problem.

• Each electron interacts with an effective average field produced by all of the other electrons • Non-linear set of coupled differential equations

 

Density Functional Equations

  2 

V

(

r

)  

j

(

r

) 

E j



j

(

r

) Looks linear but...

V

(

r

) through: depends on the charge density

V

(

r

)    (

r

)

d

3

s

  

j

(

r

) 2  (

s

)

r



j



s

   (

r

)   (

r

)

d

3

s

 (

r

) 

xc

(  (

r

)) 

DFT solution approach

• Expand the wave-functions in a basis set: 

j

(

r

)  

C l j



l

(

r

)

l

• Matrix eigenvalue-eigenvector problem: 

H jl C l j l

   *

C l j

 

kj l

• Iterative solution to “self-consistency”

(i.e. output V(r) coincides with input)



EC j j



Popular implementations

• Plane wave basis functions (Fourier Series): 

j

(

r

)  1 exp(

ik j



r

)

V O

(

N

) scaling – Benefit: easy to code, sophisticated non-linear response calculations possible  • Localized “atomic-like” basis functions  

j

(

r

) 

a j

(

r

) - exponential distance decay for insulators - power law distance decay for - metals

Contrasting Implementations

• Abinit: www.abinit.org

– Very sophisticated array of calculated properties – Calculations become prohibitive for more than a few dozen atoms

O

(

N

3 ) • VASP (Vienna Ab-Initio Simulation Package) – Less sophisticated by much faster – few hundred atoms possible

O

(

N

3 )  Simulations with Thousands of Atoms) – O(N) scaling: fast but less sophisticated 

Public Access

• Many codes are freely available: go to http://psi-k.dl.ac.uk/data/codes.html

for a list of more than 20 • Most codes still not user-friendly and take months to years to master

The Brick Wall!!

• All of these methods run out of steam very quickly in terms of run time and memory • Calculations with scaling take days or weeks to run!!

O

(

N

3 ) memory bottlenecks 

O

(

N

) • Materials Science simulations require thousands of atoms for thousands of time steps

Key Algorithms

• For plane wave based codes: the

Fast Fourier Transform

– We have gained factor’s of 4 improvement in speed and storage using Conformal Computing – A number of new developments are being implemented for further increases • Matrix diagonalization routines for very large matrices

Conformal Computing

• Density Functional Calculations are an ideal setting for Conformal Computing!

• In fact: any array (matrix) based computational setting is ripe for Conformal Computing • Why? Conformal Computing eliminates temporary arrays and un-necessary loops!

Opportunities

• Current electronic band structures fairly fast (on the order of one hour):

Contrasting: electron-phonon

• Electron-phonon calculations: on the order of 1 day for small systems • Superconductivity in “conventional” materials determined by the electron phonon interaction • Aluminum (1 atom) takes roughly 1 day of computing • Imagine several dozen atoms

O

(

N

3 ) 

Electron-Phonon improvements

• Many quantities currently written to files then later combined • The size and number of these files is becoming prohibitively expensive • Opportunities for parallelization of integrals • Opportunities to eliminate temporaries through the use of direct indexing

Grid Computing

• Even with highly optimized code (which is still a way off) there is always a need for more and more resources • For example: electron-phonon calculations involve dozens of separate calculations that could be run on independent machines • Grid computing allows many independent calculations to be run in parallel

Grid Computing: First Steps

• QMolDyn GAT: a template for submitting Density Functional Calculations over the grid • Vision: QMolDyn will eventually have a variety of codes (modules) • Presently: Siesta ( ) running on the grid, 8, 16, 32, 64, 128, 256, 512- atom systems 

Summary / Conclusions

• There is a great demand for large-scale array (matrix) based calculations in materials science • Quantum calculations are increasingly important for Materials Science, Chemistry and Physics • Grid computing combined with Conformal Computing techniques is very promising