Investigation of Hot Spot Region in XIAP Inhibitor Binding Site by Fragment Molecular Orbital Method

X-linked inhibitor of apoptosis protein (XIAP) is an important regulator of cancer cell survival whose BIR3 domain (XIAP-BIR3) recognizes the Smac N-terminal tetrapeptide sequence (AVPI), making it an attractive protein-protein interaction (PPI) target for cancer therapies. We used the fragment molecular orbital (FMO) method to study the binding modes and affinities between XIAP-BIR3 and a series of its inhibitors (1–8) that mimic the AVPI binding motif; the inhibitors had common interactions with key residues in a hot spot region of XIAP-BIR3 (P1–P4 subpockets) with increased binding affinity mainly attributed to specific interactions with the P1 and P4 subpockets. Based on the structural information from FMO results, we proposed a novel XIAP natural product inhibitor, neoeriocitrin 10, which was derived from our preciously reported XIAP-BIR3 inhibitor 9, can be used as a highly potent candidate for XIAP-BIR3 inhibition. We also performed pair interaction energy decomposition analysis to investigate the binding energies between specific binding residues and individual ligands, showing that the novel natural product neoeriocitrin 10 had a higher binding affinity than epicatechin gallate 9. Molecular docking and dynamics simulations were performed to explore the mode of binding between 10 and XIAP-BIR3, demonstrating that 10 binds more strongly to the P1 and P4 pockets than 9. Overall, we present a novel natural product, neoeriocitrin 10, and demonstrate that the FMO method can be used to identify hot spots in PPIs and design new compounds for XIAP inhibition.