# Day 2: "Beyond-DFT" calculations in practice with JDFTx + BGW

## Introduction

<ul>
This tutorial provides an example of how JDFTx and BerkeleyGW can be combined to calculate RPA+EXX@DFT total energies. BerkeleyGW is needed for this method because JDFTx cannot calculate the necessary RPA correlation energies. For this tutorial, we will be comparing the DFT and RPA adsorption energy of an H atom on a Pt(111) surface at full coverage. If you are not familiar with the idea of an adsorption energy, it is the energy gain/loss that occurs when some species (here: the H atom) bonds with a surface (here: Pt(111)). To calculate an adsorption energy of the H atom, $\mathrm{E_{ads,H}}$, we use the following equation:
</ul>
<ul><ul>
$\mathrm{E_{ads,H} = E_{Pt(111)+H} - (E_{Pt(111)} + \frac{1}{2} E_{H_2}})$
</ul></ul>
<ul>
which compares the energy of the H atom adsorbed on Pt(111), $\mathrm{E_{Pt(111)+H}}$, to the combined energy of the isolated species and "clean" surface, $\mathrm{E_{Pt(111)} + \frac{1}{2} E_{H_2}}$. Because the H atom will typically exist as a diatomic molecule, we use the H$_2$ molecule energy as a reference and divide by 2, although other reference schemes exists. Here, we will calculate the energy terms above at the DFT and RPA+EXX@DFT levels of theory. 
</ul>
<ul>
We note that this tutorial is only a very brief overview of the capability of BerkeleyGW and the theory behind these calculations. If you are interested in running calculations like this, it is highly recommended to read more about the BerkeleyGW code:
</ul>
<ul>
    
1. [BerkeleyGW website](https://berkeleygw.org/)
2. Jack Deslippe, Georgy Samsonidze, David A. Strubbe, Manish Jain, Marvin L. Cohen, and Steven G. Louie, [“BerkeleyGW: A Massively Parallel Computer Package for the Calculation of the Quasiparticle and Optical Properties of Materials and Nanostructures”](https://www.sciencedirect.com/science/article/pii/S0010465511003912?via%3Dihub), Comput. Phys. Commun. 183, 1269 (2012) 

</ul>
<ul>
and the ACDFT RPA formalism for semiconductors and metallic systems. 
</ul>

## Workflow overview
<ul>
RPA+EXX@DFT calculations (RPA+EXX calculations based off a DFT starting point) require multiple steps, which are listed below:
</ul>
<ul>
    
1. Calculate DFT charge densities using DFT-relaxed structures. This step is typically run with a limited number of unoccupied bands to save computational expense. This step occurs in the 1-dft/ directories.
2. Determine the k/q-grid that will be used to calculate EXX and the RPA correlation energy. This step occurs in the 2-generatekgrid/ directories. 
3. Calculate EXX starting from the DFT charge density and structure. This step occurs in the 3-exx/ directories. 
4. Calculate the eigenvalues of many additional unoccupied bands starting from the DFT charge density and structure. This step occurs in the 4-wfn/ directories. 
5. Calculate the RPA correlation energy using the wavefunction generated in Step 4. There are several jobs that must be run for this step so that we can perform the needed basis set extrapolation. This step occurs in the 5-rpa/ directories. 
6. Calculate the final RPA+EXX@DFT total energies using DFT and RPA+EXX@DFT.
</ul>
<ul>
Here, steps 1, 3, and 4 will be run using JDFTx and step 5 will be run using BerkeleyGW. Step 2 only requires running a simple script provided by BerkeleyGW. Step 6 is a simple post-processing step that can be done using Python or Excel.
</ul>
<ul>
In this tutorial, these steps can be performed by running the various executables in the pt/, pt+h/, and h2/ directories in numerical order by their prefix (run the script starting with 01_ first, 02_ second, etc.). 
</ul>
<ul>
To start, please descend into the pt/ directory. We will first calculate the DFT and RPA+EXX@DFT total energies for the clean Pt(111) surface. Once you have completed all 6 of the below Steps, please return to Step 1, navigate to the pt+h/ directory and repeat all steps again. After completing this, navigate to the h2/ directory and repeat all steps yet again before moving on to the next section. The update_submitfiles.sh utility has been included only for convenience if you need to update all of the module submit.sh files for some reason. You likely won't need to use it. 
</ul>

## Step 0 - Open the README.md file and a terminal window
<ul>
This tutorial will be run using the terminal and Jupyter notebooks. Please keep this README.md preview open and open a new terminal tab that you can use to run code on the terminal command line. These commands can be copied and pasted into the command line and will be formatted like this:
</ul>
<ul>
    
```bash
./XX_example_script.sh
```

</ul>
<ul>
Please open the needed .ipynb files by double-clicking on them in the the file navigator to the left. If Jupyter ever needs you to select a kernel due to a restart, please select 'BEAST Python' from the list. All .ipynb should already have this set (which you can see in the top right corner). 
</ul>
<ul>
Note that any scripts with the name format XX_run_???.sh submit jobs to the queue. Please do not run these scripts multiple times in a row. If, for some reason, you do need to resubmit a job, please check that the previously submitted job has finished with "sq" (the alias of "squeue -u YOUR_USERNAME" you should have set earlier from the SETUP document). You can cancel jobs with "scancel THE_JOB_ID_NUMBER".
</ul>

## Step 1 - DFT job
<ul>
Descend into 1-dft/ in both your terminal (cd 1-dft) and the file navigator to the left. First, let's visualize the structure we are running. Open the 01a_visualize_structure.ipynb jupyter notebook in a new tab. Then, click on the code block and hit "Shift+Enter" to run it. You should see a structure render (you may need to zoom in or out some). You can see this is a very small surface with only 3 atoms unique atoms (repeated periodically 3 times along x and y for visualization purposes). 
</ul>
<ul>
Once you are done looking at the structure, run the following terminal command:
</ul>
<ul>
    
```bash
./01b_run_dft.sh
```

</ul>
<ul>
Verify that this job has run sucessfully by examining the output file (name: "out"). You should see the usual timing lines printed at the end of the file. Additionally, the total energy close to the end should be: F = -363.9107 Ha. The input tags used here closely mirror those used in previous JDFTx DFT tutorials. This calculation uses a 20 Ha planewave cutoff. Note that the 4x4x1 k-grid we are using here is underconverged to allow these calculations run quickly. Metallic systems require significantly higher grids (12x12x1 for this system would be much better, for example) for convergence. Why is this? Look at the Lowdin population analysis at the end of the out file. How does it compare to the results from the previous tutorials? Do these systems have a net moment? Which atoms are oxidized or reduced?
</ul>
<ul>
If the job has run successfully, navigate back up a directory (cd ../) on both your terminal and file navigator to the left. 
</ul>
    
## Step 2 - BerkeleyGW k/q-grid generation
<ul>
Descend into 2-generatekgrid/ and run:
</ul>
<ul>
    
```bash
./02_gen_kgrid.sh
```

</ul>
<ul>
Ignore all the Lmod/MODULEPATH and conda messages, these happen because the script is (un)loading some modules to get into the right environment that conflict with JDFTx. This is a simple script that runs the kgrid.x executable provided by BerkeleyGW. kgrid.x takes kgrid.in as input, produces a list of k/q-points in kgrid.out, and logs relevant symmetry information used to generate kgrid.out in kgrid.log. 
</ul>
<ul>
This step is necessary because BerkeleyGW requires the user to provide an explicit list of k/q-points rather than specifying a grid folding like typically done in DFT calculations. RPA and EXX calculations do not necessarily need to be run on the same grid folding as DFT calculations but will for this tutorial. The kgrid.out file contains the list of points that will be provided to BerkeleyGW. However, the list needs to be converted into the format expected by JDFTx by adding the "kpoint" tag to each point and normalizing all of the k-point weights in the right column to a sum of 1. This has already been done and the kgrid.jdft file contains this information. It will be copied to the 3-exx/ and 4-wfn/ directories by scripts in Steps 3 and 4.  
</ul>
<ul>
Navigate back up a directory. 
</ul>

## Step 3 - JDFTx EXX calculation
<ul>
Now we are ready to set up the JDFTx EXX calculation. Descend into 3-exx/ and run:
</ul>
<ul>
    
```bash
./03a_setup_exx.sh
```
   
</ul>
<ul>
This script copies input and output files from Step 1 for reuse. The EXX calculation is run at fixed DFT charge density to save computational expense. 
</ul>
<ul>
Next, run:
</ul>
<ul>
    
```bash
./03b_run_exx.sh
```

</ul>
<ul>
to start the job. It will take about 90 seconds to run. While the job is running, look at the input file for this job (name: "in") and see what has changed as compared to the "in" file in 1-dft/. The important things to note are:
</ul>
<ul>

1. Fewer dump tags are used because we only need 1 number printed in the output file. 
2. The k-grid tags have changed to not expand the k-grid automatically because we read in kgrid.jdft (the "include kgrid.jdft" tag). 
3. We have added a new tag called "bgw-params". This tag controls what we print for RPA+EXX@DFT calculations. Here, we specify that we want the EXX energy using "rpaExx yes". 

</ul>
<ul>
Verify that this job has run sucessfully by examining the output file (name: "out"). The energy here should be: EXX(RPA) = -31.6517 Ha. If you have run hybrid functional calculations such as PBE0 or HSE06, how are those calculations similar to this calculation?
</ul>
<ul>
Navigate back up a directory. 
</ul>

## Step 4 - JDFTx empty state generation
<ul>
Now we are ready to set up the JDFTx empty state generation calculation for later use in Step 5. Descend into 4-wfn/ and run:
</ul>
<ul>
    
```bash
./04a_setup_wfn.sh
```
    
</ul>
<ul>
This script copies input and output files from Step 1 for reuse. The wavefunction generation calculation is run at fixed DFT charge density. 
</ul>
<ul>
Next, run:
</ul>
<ul>
    
```bash
./04b_run_wfn.sh
```
    
</ul>
<ul>
to start the job. It will take about 2-4 minutes to run. While the job is running, note that the input file for this job (name: "in") is very similar to the one in 3-exx/. The only change is for the bgw-params tag, which now specifies "nBandsDense 500". This input tells JDFTx to generate a wavefunction with 500 empty bands, rather than the 34 used in the DFT calculation. 
</ul>
<ul>
Verify that this job has run sucessfully by examining the output file (name: "out"). To check that the wavefunction was generated correctly, run:
</ul>
<ul>
    
```bash
./04c_analyze_wfn.sh
```
    
</ul>
<ul>
This script runs a BerkeleyGW utility that compiles important information in the wavefunction into a human-readable condensed format file called "wfn_info.txt". Open this file and examine some of the information there if you wish. In the K-POINTS section, you should see that 500 bands are present and you should see the same list of points listed in kgrid.jdft. The ifmax field lists the highest occupied band for each k/q-point, are these results what you would expect for this metallic system? How would they differ for an insulator that has a band gap?
</ul>
<ul>
Navigate back up a directory. 
</ul>

## Note about solvated RPA calculations
<ul>
Although we are running a vacuum calculation for this tutorial, JDFTx is able to generate a file containing data about the implicit solvent's polarizability using the bgw-params tag. This file can be read in by BerkeleyGW and added to the polarizability calculated for the system of interest. The fluid polarizability modifies the screened environment of the system and thus will affect calculated properties such as the RPA correlation energy. This feature is currently being developed and tested further and a tutorial will likely be provided at next year's workshop. If you are interested in this feature please reach out to us! We have already implemented this feature to enable solvated GW approximation calculations. If you are interested in learning more about this feature please see the 5-solvated_gw workshop module and our paper entitled "Impact of solvation on the GW quasiparticle spectra of molecules", which has been recently accepted into the Journal of Applied Physics and should be made available in final published form within the next couple of weeks. 
</ul>
    
## Step 5 - BerkeleyGW RPA correlation energy calculation
<ul>
Descend into 5-rpa/ and run:
</ul>
<ul>
    
```bash
./05a_setup_rpa.sh
./05b_run_rpa.sh
```
    
</ul>
<ul>
first in order to get this job in the queue while you read the rest of this step. It will take about 10 minutes to finish. 
</ul>
<ul>
    
### RPA background
</ul>
<ul><ul>
The RPA correlation energy is calculated using the following equation within the adiabatic connection fluctuation-dissipation theorem (ACFDT):
</ul></ul>
<ul><ul><ul>
$E_\mathrm{corr,RPA} = \int_{0}^{\infty}\frac{d\omega}{2\pi}Tr\{ln[1-\nu\chi^{KS}(i\omega)]+\nu\chi^{KS}(i\omega)\}$
</ul></ul></ul>
<ul><ul>
where $\nu$ is the local potential and $\chi^{KS}(i\omega)$ is the response function (polarizability) of the noninteracting Kohn-Sham system at imaginary frequencies. BerkeleyGW calculates $\chi^{KS}(i\omega)$ on a specified set of frequency grid points using a basis set. As an aside, the response function can be used to calculate a material's dielectric matrix, $\epsilon$, the namesake of the files in this step. The size of this basis set for the response function is determined by the value of the screened cutoff. Unfortunately, the response function converges extremely slowly with the size of the basis set and so is generally calculated several times using different screened cutoffs and extrapolated to infinite basis set size (see next Step for method). Among other reasons, this is one reason why RPA calculations are significantly more expensive than normal DFT calculations. 
</ul></ul>
<ul>

### RPA screened cutoff extrapolation
</ul>
<ul><ul>
In this tutorial, the several subdirectories in the 5-rpa/ directory (6.0, 6.5, 7.0, 7.5, 8.0, and 8.5) are named after the screened cutoff used in the run (6.0 Ryd, 6.5 Ryd, etc.). Consequently, each of these 6 subdirectories is a separate BerkeleyGW calculation. The only differences between each calculation are the values specified by the epsilon_cutoff and number_bands tags in the input files given to BerkeleyGW (name: "epsilon.inp"). Run:
</ul></ul>
<ul><ul>
    
```bash
./05c_compare_inputfiles.sh
```
    
</ul></ul>
<ul><ul>
to see how these tags change for each run. The value provided to number_bands can be calculated using the gsphere.py BerkeleyGW utility. However, this will not be covered here for brevity. 
</ul></ul>
<ul>

### BerkeleyGW input file and job setup
</ul>
<ul><ul>
While you are looking at the epsilon.inp files, open the 6.0/epsilon.inp file to see the other information specified there. Here are some highlights: cell_slab_truncation truncates the Coulomb interaction perpendicular to the Pt(111) slab. The fermi_level tags specify the Fermi level of this system in eV (the mu value in Ha at the end of the 1-dft/out file has been converted). The occ_broadening tag specifies the same broadening (in eV) set in elec-smearing in the JDFTx "in" file in 1-dft/. These values can be converted to Ryd by multiplying by 2.0 Ryd/Ha and converted to eV by multiplying by 2.0 Ryd/Ha * 13.605692525 eV/Ryd. The q-point grid almost matches kgrid.jdft from earlier steps. The $q_{0}$ point is replaced by the next closest point in order to account for the response function going to infinity for metallic systems as q $\rightarrow$ 0. The last tags specify the frequency grid used to calculate the polarizability and other internal tags specific to BerkeleyGW. 
</ul></ul>
<ul><ul>
Next, each screened cutoff job reads in the wavefunction generated in the previous step. However, because this wavefunction is generally dozens or hundreds of gigabytes, we typically create a symbolic link to it to save space. The symbolic links were created by running the ./05a_setup_rpa.sh script. If you descend into 6.0/ and run "ls -l" in the terminal, you will see that the file named "WFN.h5" points to the original wavefunction. 
</ul></ul>
<ul><ul>
Next, look at the submit.sh file in 5-rpa/. For this tutorial, we are running each screened cutoff job in serial. However this does not have to generally be the case, as each screened cutoff can be run as a separate job and may be very expensive on its own. 
</ul></ul>
<ul>

### Results analysis
</ul>
<ul><ul>
Finally, after a few minutes, some of the screened cutoff jobs should have run. Check that the 6.0/ job ran by opening the output file (name: "out") and checking if the RPA energy at the end is: RPA energy (Ha) = -1.4248. If so, the run is working and you just have to wait for the other screened cutoff jobs to finish. 
</ul></ul>
<ul><ul>
To extrapolate the 6 RPA energies we have at finite screened cutoffs to infinite cutoff, we can use the Lindhard function, which has been shown to be a good model for this method:
</ul></ul>
<ul><ul><ul>
$\mathrm{E_{corr,RPA}[E_{cut}^\chi] = E_{corr,RPA}^\infty + \frac{A}{(E_{cut}^\chi)^{3/2}}}$
</ul></ul></ul>
<ul><ul>
where $E_{corr,RPA}[E_{cut}^\chi]$ is the RPA energy at screened cutoff $E_{cut}^\chi$, $E_{corr,RPA}^\infty$ is the final energy we are interested in, and $A$ is a fitting coefficient. Note that because the screened cutoffs are in the denominator, as cutoff approaches infinity, the x value $\rightarrow$ 0, meaning that the y-intercept of the fitted line is the infinite cutoff limit. 
</ul></ul>
<ul><ul>
To carry out this extrapolation, open the 05d_extrapolate_RPAenergies.ipynb jupyter notebook file. Note that we are using Ha for the energy units because JDFTx provides energies in Ha. This does not affect the linear extrapolation. Click in the first code block and run the code. You should see the code print the RPA energy lines from your output files and summarize the values. Move to the next code block and run it. This code block performs the linear regression, prints the y-intercept and R^2 value, and writes the RPA extrapolated energy to a text file (name: "RPAextrap.dat"; this file may take a few seconds to show up in your file list to the left). If you have Excel or an equivalent program installed on your computer, it is recommended to copy the screened cutoffs and RPA energies associated with each into the program to check the extrapolation manually. Move to the next code block and run it. This code block plots the results of the extrapolation so you can see what is happening. Once you run all of this code, you should have a final RPA extrapolated energy of -1.9570 Ha and a plot that looks like this:
    
![Pt RPA extrapolation](pt/5-rpa/Pt_reference_RPAextrapolation.png)
    
</ul></ul>
<ul><ul> 
Navigate back up a directory.
</ul></ul>

## Step 6 - RPA+EXX@DFT total energy calculation
<ul>
Now that we have calculated all of the needed energy terms, we can calculate a RPA+EXX@DFT total energy using the equation: 
</ul>
<ul><ul>
$\mathrm{E_{RPA} = E_{DFT} - E_{XC} + E_{XX} + E_{corr,RPA}}$
</ul></ul>
<ul>
where $E_{DFT}$ is the total DFT energy, $E_{XC}$ is the DFT exchange-correlation energy, $E_{XX}$ is the exact exchange energy, and $E_{corr,RPA}$ is the RPA correlation energy. 
</ul>
<ul>
These energy terms can be found by opening the 06_calculate_RPAenergy.ipynb jupyter notebook file. Run the code block. You should see all of the relevant energy components print. A summary file (name: "totalenergies.dat") will also be written. You should get a total RPA energy of -362.8054 Ha.
</ul>
<ul>
Navigate back up a directory. 
</ul>
<ul>
If you have not yet completed all 6 Steps for the pt/, pt+h/, and h2/ directories, please return to Step 1 and repeat this process for all 3 systems. The next section cannot be completed until these calculations are finished. The pt+h/ system jobs will take about as long as the pt/ ones. The h2/ system jobs will be much shorter, with the RPA job only taking a couple of minutes. However, note that for the h2/ job, we generate 900 bands in 4-wfn/ instead of 500. This is because the volume of cell made by the lattice vectors in this job (and thus number of planewaves needed) is larger than the Pt surface calculations. 
</ul>

# Final adsorption energy calculations
<ul>
Now that you have completed calculating the total energies of all 3 systems using DFT and RPA+EXX@DFT, we are finally ready to compare the adsorption energies of each method. Open the final_compare_Eads.ipynb jupyter notebook and run the first code block. This code looks for the 3 "totalenergies.dat" files and calculates adsorption energies for DFT and RPA+EXX@DFT using the very first equation in this README. You should get DFT and RPA+EXX@DFT adsorption energies of -0.21 eV (-0.0077 Ha) and -0.37 eV (-0.0134 Ha), respectively. 
</ul>
<ul>
As a final important note, these RPA+EXX@DFT calculations were run at fixed structures found by DFT ionic relaxations. Thus, they are non-self-consistent and would require RPA forces to achieve full self-consistency. 
</ul>
<ul> 
Now, here are some questions to consider or read further about if you are interested:
</ul>
<ul>
    
1. Besides the structure issue, what are other reasons DFT and RPA+EXX@DFT would differ in their adsorption energies? 
2. RPA+EXX@DFT includes long-range interactions better than DFT. Would you expect this system to be particularly sensitive to long-range forces? What are some systems you know would be sensitive to long-range van der Waals interactions? 
3. The number of bands used in the RPA correlation energy calculations increase as more DFT planewaves are used in the calculation. How do the number of planewaves scale with system size and what does this suggest about RPA computational expense?
</ul>

# BONUS EXERCISES
<ul>
If you would like additional systems to run, try actually running calculations to get numbers associated with question 3. Copy the 1-scf/ directory for pt/ into a few new directories and increase/decrease the planewave cutoff in the "in" file in increments of 5. grep for "average nbasis" (average planewaves across all k-points) in the JDFTx "out" file and plot each planewave cutoff vs this value. 
</ul>
<ul>
Similarly, try copying the 5-rpa/8.5/ directory into a new directory and increase the screened cutoff to 20 Ryd while keeping everything else (including which file WFN.h5 is linked to) the same. Change the value after number_bands to be an appropriate integer (hint: it should be slightly more than 1000). Use your results from the previous paragraph where you got different numbers of planewaves by modifying the JDFTx planewave cutoff (hint: try including 6 through 8 Ryd in your above testing to help see this connection). Modify submit.sh to only run this single new screened cutoff. Do you get an error? Why?
</ul>
<ul>
Think about how many bands you want to include now as compared to what you generated previously in 4-wfn. Copy 4-wfn to a new directory and modify 4-wfn-new/in to be more appropriate for your larger screened cutoff (use a few more bands than what you set number_bands to; there are internal BerkeleyGW checks that need the wavefunction to have at least 1 extra band than number_bands). Update the 5-rpa/20/ WFN.h5 link. Rerun the 5-rpa/20/ job. How much slower is each q-point now? This is one of the reasons why few people apply RPA towards large systems; the cost grows to be intractable very quickly. 
</ul>

