Res ipSAE loquunt: What's wrong with AlphaFold's ipTM score and how to fix it

Abstract

AlphaFolds ipTM metric is used to predict the accuracy of structural predictions of protein-protein interactions (PPIs) and the probability that two proteins interact. Many AF2/AF3 users have experienced the phenomenon that if they trim full-length sequence constructs (e.g. from UniProt) to the interacting domains (or domain+peptide), their ipTM scores change, even though the structure prediction of the interaction is unchanged. The reason this happens is due to the mathematical formulation of ipTM in AF2/AF3, which scores the interactions of whole chains. When one chain is a single ordered domain but the other contains lots of disorder or irrelevant domains, ipTM is higher in the presence of those residues than it is when they are trimmed. If both chains in a PPI complex contain large amounts of disorder or accessory domains that do not form the primary domain-domain or domain/peptide interaction, the ipTM score can be significantly lower in the presence of those regions than when they are trimmed. The score then does not accurately represent the accuracy of the structure prediction nor whether the two proteins actually interact. We have solved this problem by: 1) including only residue pairs in the ipTM metric that have good predicted aligned error (PAE) scores; 2) by adjusting the d0 parameter (a function of the length of the query sequences) in the TM score equation to include only the number of residues with good interchain PAEs to the aligned residue; and 3) using the PAE value itself and not the probability distributions over the aligned error to calculate the pairwise residue-residue pTM values that go into the ipTM calculation. The first two are crucial in calculating high ipTMs for domain-domain and domain-peptide interactions even in the presence of many hundreds of residues in disordered regions and/or accessory domains. The third allows us to require only the common output json files of AF2 and AF3 (including the server output) without having to change the AlphaFold code and without affecting the accuracy. It also works on npz file output from Boltz. We show in a benchmark that the new score, called ipSAE (interaction prediction Score from Aligned Errors), is able to separate true from false complexes more efficiently than AlphaFold2s ipTM score. The resulting program is freely available at https://github.com/dunbracklab/IPSAE.