CASP16 Call for Targets

YH
Yu He Liang
Fri, Apr 5, 2024 4:27 PM

Dear PDB-l

CASP (Critical Assessment of Structure Prediction) experiments are held
every two years. Recent rounds have seen dramatic increases in modeling
accuracy, resulting from the introduction of deep learning methods: In
2018, for the first time, the folds of most proteins were correctly
computed [1]; in 2020, the accuracy of many computed protein structures
rivaled that of the corresponding experimental ones [2]; in 2022, there was
an enormous increase in the accuracy of protein complexes [3].

We have seen the beginning of what deep learning methods may achieve in
structural biology. In addition to further increases in the accuracy of
protein complexes, methods are being developed for RNA structures, organic
ligand-protein complexes, and for moving beyond single macromolecular
structures to compute conformational ensembles. Accurate computational
methods together with experimental data also offer the prospect of probing
previously inaccessible biological systems. CASP has expanded its scope to
provide critical assessment in all these areas.

CASP is only possible with the generous participation of the experimental
structural biology community in providing suitable targets: A total of over
1100 targets have been obtained over the previous CASP rounds. We are now
requesting targets for the 2024 CASP16 experiment. We need challenge
targets in the following areas:

Single protein structures: The 2020 and 2022 CASPs showed that, so far,
Alphafold2 and methods built around it are by far the most accurate [4].
But there are limitations, particularly for some proteins where only a
shallow sequence alignment is available and for very large proteins (more
than 1000 amino acids). The best results also require substantial amounts
of computing resources, well beyond that of the AlphaFold2 default
settings. Many new methods are continuing to appear and these may remove
some of the remaining difficulties. All types of protein targets are
needed, but especially those with shallow sequence alignments, without
structural templates, and large proteins.

Protein complexes: In the 2022 CASP15, advanced deep learning methods were
applied to protein complexes for the first time [5]. The result was a huge
improvement in accuracy compared with classical docking approaches. But
overall, the results are still not at the level achieved for single
proteins. So, in CASP16 we need all sorts of targets in this area so as to
determine progress since then. We particularly need complexes where there
is no evolutionary information across the protein-protein interfaces, for
example, antibody-antigen complexes. (This CASP category is conducted in
close collaboration with our colleagues at CAPRI - Critical Assessment of
protein interactions [6]).

Nucleic acid structures and complexes: In recognition of the major role
nucleic acid structures and complexes play in biology, CASP now includes
this class of target. A number of papers claiming successful RNA structure
computation using deep learning methods have been published, but those
participating in the 2022 CASP RNA category performed less well than
classical approaches, and no methods were able to effectively address the
two RNA protein-complexes included [7]. CASP needs a wide variety of RNA,
DNA, and complexes as targets to see if this situation has changed. (This
CASP category is conducted in close collaboration with RNApuzzles [8]).

Organic ligand-protein complexes: This area is of major importance for
computer-aided drug discovery. Earlier, there have been community
experiments to assess the accuracy of methods, particularly SAMPL, CSAR,
D3R, and a new one, CACHE, has recently started (http://cache-challenge.org
). These challenges have drawn strong international participation from
researchers in both academia and industry. Here too, a number of promising
deep learning papers have appeared, but in the 2022 CASP15 pilot, classical
methods were still superior [9]. So, we need appropriate targets to see if
progress has been made since. Ideally, these should be sets of
three-dimensional protein-ligand complexes from drug discovery projects,
but single targets would also be appreciated. Additionally, where
available, we will assess non-structural quantities such as affinities or
affinity rankings and other properties of pharmaceutical interest when
these are available (small molecule pKs, and DMPK related properties).

Ensembles of macromolecule conformations: It is now widely recognized that
proteins and nucleic acids often adopt multiple conformations that can
underpin their functions. In these cases, considering only a single protein
or RNA conformation may be a significant oversimplification. The 2022
CASP15 included a pilot experiment to assess methods for computing multiple
conformations, with encouraging results [10], but with limitations imposed
by the available experimental data. For 2024, we seek not only cases of
multiple experimental three-dimensional structures for the same
macromolecule but also other types of data that might be used for
assessment of computed conformation ensembles such as cryoEM, NMR, X-ray
crystallography, SAXS, and/or cross-link data.

Integrative modeling: The more powerful computational methods open up new
possibilities for combination with sparse or low-resolution experimental
data to investigate previously inaccessible biological structures and
machines. CASP is interested in exploring these possibilities and so
requests experimentally difficult targets where structure has nevertheless
been obtained. In appropriate cases, we expect to be able to collaborate
with other experimental groups to provide appropriate data from NMR,
cross-linking or SAXS.

There are three avenues to contribute a target to CASP:

(preferrable) Submit directly to CASP through our web-interface

https://predictioncenter.org/casp16/targets_submission_form.cgi (requires a
quick registration at https://predictioncenter.org/login.cgi if you do not
have an account with us).

Email to targets@predictioncenter.org with your target suggestions or to
discuss any questions.
3.

Submit your structure to the PDB (on-hold) and designate it as a CASP
target through PDB’s submission interface.

The timeline for the 2024 CASP requires that targets are submitted starting
now and until July 1. We would like to hear from you as soon as possible if
you may have something suitable or have suggestions about other target
sources. In order to maintain rigor, the experimental data for a target
must not be publicly available until after computed structures have been
collected. For assessment, CASP requires the experimental data by August
15, but the data can remain confidential after that. Target providers are
invited to contribute to papers [11-15] for a special CASP issue of the
journal Proteins.

CASP organizers: John Moult, Krzysztof Fidelis, Andriy Kryshtafovych,
Torsten Schwede, Maya Topf

References

Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XIII.
Proteins 2019;87(12):1011-1020.
2.

Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XIV.
Proteins 2021;89(12):1607-1617.
3.

Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XV.
Proteins 2023;91(12):1539-1549.
4.

Simpkin AJ, Mesdaghi S, Sanchez Rodriguez F, Elliott L, Murphy DL,
Kryshtafovych A, Keegan RM, Rigden DJ. Tertiary structure assessment at
CASP15. Proteins 2023;91(12):1616-1635.
5.

Ozden B, Kryshtafovych A, Karaca E. The impact of AI-based modeling on
the accuracy of protein assembly prediction: Insights from CASP15. Proteins
2023;91(12):1636-1657.
6.

Lensink MF, Brysbaert G, Raouraoua N, Bates PA, Giulini M, Honorato RV,
van Noort C, Teixeira JMC, Bonvin A, Kong R, Shi H, Lu X, Chang S, Liu J,
Guo Z, Chen X, Morehead A, Roy RS, Wu T, Giri N, Quadir F, Chen C, Cheng J,
Del Carpio CA, Ichiishi E, Rodriguez-Lumbreras LA, Fernandez-Recio J,
Harmalkar A, Chu LS, Canner S, Smanta R, Gray JJ, Li H, Lin P, He J, Tao H,
Huang SY, Roel-Touris J, Jimenez-Garcia B, Christoffer CW, Jain AJ, Kagaya
Y, Kannan H, Nakamura T, Terashi G, Verburgt JC, Zhang Y, Zhang Z, Fujuta
H, Sekijima M, Kihara D, Khan O, Kotelnikov S, Ghani U, Padhorny D, Beglov
D, Vajda S, Kozakov D, Negi SS, Ricciardelli T, Barradas-Bautista D, Cao Z,
Chawla M, Cavallo L, Oliva R, Yin R, Cheung M, Guest JD, Lee J, Pierce BG,
Shor B, Cohen T, Halfon M, Schneidman-Duhovny D, Zhu S, Yin R, Sun Y, Shen
Y, Maszota-Zieleniak M, Bojarski KK, Lubecka EA, Marcisz M, Danielsson A,
Dziadek L, Gaardlos M, Gieldon A, Liwo A, Samsonov SA, Slusarz R, Zieba K,
Sieradzan AK, Czaplewski C, Kobayashi S, Miyakawa Y, Kiyota Y,
Takeda-Shitaka M, Olechnovic K, Valancauskas L, Dapkunas J, Venclovas C,
Wallner B, Yang L, Hou C, He X, Guo S, Jiang S, Ma X, Duan R, Qui L, Xu X,
Zou X, Velankar S, Wodak SJ. Impact of AlphaFold on structure prediction of
protein complexes: The CASP15-CAPRI experiment. Proteins
2023;91(12):1658-1683.
7.

Das R, Kretsch RC, Simpkin AJ, Mulvaney T, Pham P, Rangan R, Bu F,
Keegan RM, Topf M, Rigden DJ, Miao Z, Westhof E. Assessment of
three-dimensional RNA structure prediction in CASP15. Proteins
2023;91(12):1747-1770.
8.

Magnus M, Antczak M, Zok T, Wiedemann J, Lukasiak P, Cao Y, Bujnicki JM,
Westhof E, Szachniuk M, Miao Z. RNA-Puzzles toolkit: a computational
resource of RNA 3D structure benchmark datasets, structure manipulation,
and evaluation tools. Nucleic Acids Res 2020;48(2):576-588.
9.

Robin X, Studer G, Durairaj J, Eberhardt J, Schwede T, Walters WP.
Assessment of protein-ligand complexes in CASP15. Proteins
2023;91(12):1811-1821.

  1. Kryshtafovych A, Montelione GT, Rigden DJ, Mesdaghi S, Karaca E, Moult
    J. Breaking the conformational ensemble barrier: Ensemble structure
    modeling challenges in CASP15. Proteins 2023;91(12):1903-1911.
  2. Kretsch RC, Andersen ES, Bujnicki JM, Chiu W, Das R, Luo B, Masquida B,
    McRae EKS, Schroeder GM, Su Z, Wedekind JE, Xu L, Zhang K, Zheludev IN,
    Moult J, Kryshtafovych A. RNA target highlights in CASP15: Evaluation of
    predicted models by structure providers. Proteins 2023;91(12):1600-1615.
  3. Alexander LT, Durairaj J, Kryshtafovych A, Abriata LA, Bayo Y, Bhabha
    G, Breyton C, Caulton SG, Chen J, Degroux S, Ekiert DC, Erlandsen BS,
    Freddolino PL, Gilzer D, Greening C, Grimes JM, Grinter R, Gurusaran M,
    Hartmann MD, Hitchman CJ, Keown JR, Kropp A, Kursula P, Lovering AL,
    Lemaitre B, Lia A, Liu S, Logotheti M, Lu S, Markusson S, Miller MD,
    Minasov G, Niemann HH, Opazo F, Phillips GN, Jr., Davies OR, Rommelaere S,
    Rosas-Lemus M, Roversi P, Satchell K, Smith N, Wilson MA, Wu KL, Xia X,
    Xiao H, Zhang W, Zhou ZH, Fidelis K, Topf M, Moult J, Schwede T. Protein
    target highlights in CASP15: Analysis of models by structure providers.
    Proteins 2023;91(12):1571-1599.
  4. Alexander LT, Lepore R, Kryshtafovych A, Adamopoulos A, Alahuhta M,
    Arvin AM, Bomble YJ, Bottcher B, Breyton C, Chiarini V, Chinnam NB, Chiu W,
    Fidelis K, Grinter R, Gupta GD, Hartmann MD, Hayes CS, Heidebrecht T, Ilari
    A, Joachimiak A, Kim Y, Linares R, Lovering AL, Lunin VV, Lupas AN, Makbul
    C, Michalska K, Moult J, Mukherjee PK, Nutt WS, Oliver SL, Perrakis A,
    Stols L, Tainer JA, Topf M, Tsutakawa SE, Valdivia-Delgado M, Schwede T.
    Target highlights in CASP14: Analysis of models by structure providers.
    Proteins 2021;89(12):1647-1672.
  5. Lepore R, Kryshtafovych A, Alahuhta M, Veraszto HA, Bomble YJ, Bufton
    JC, Bullock AN, Caba C, Cao H, Davies OR, Desfosses A, Dunne M, Fidelis K,
    Goulding CW, Gurusaran M, Gutsche I, Harding CJ, Hartmann MD, Hayes CS,
    Joachimiak A, Leiman PG, Loppnau P, Lovering AL, Lunin VV, Michalska K,
    Mir-Sanchis I, Mitra AK, Moult J, Phillips GN, Jr., Pinkas DM, Rice PA,
    Tong Y, Topf M, Walton JD, Schwede T. Target highlights in CASP13:
    Experimental target structures through the eyes of their authors. Proteins
    2019;87(12):1037-1057.
  6. Kryshtafovych A, Albrecht R, Basle A, Bule P, Caputo AT, Carvalho AL,
    Chao KL, Diskin R, Fidelis K, Fontes C, Fredslund F, Gilbert HJ, Goulding
    CW, Hartmann MD, Hayes CS, Herzberg O, Hill JC, Joachimiak A, Kohring GW,
    Koning RI, Lo Leggio L, Mangiagalli M, Michalska K, Moult J, Najmudin S,
    Nardini M, Nardone V, Ndeh D, Nguyen TH, Pintacuda G, Postel S, van Raaij
    MJ, Roversi P, Shimon A, Singh AK, Sundberg EJ, Tars K, Zitzmann N, Schwede
    T. Target highlights from the first post-PSI CASP experiment (CASP12,
    May-August 2016). Proteins 2018;86 Suppl 1(Suppl 1):27-50.

--
YUHE LIANG, Ph.D.
Biocuration Lead Deputy, RCSB Protein Data Bank
Research Associate, Institute for Quantitative Biomedicine
Rutgers, The State University of New Jersey
174 Frelinghuysen Road, Piscataway NJ 08554
P: 848.445.4938 | E: yuhe.liang@rcsb.org

Dear PDB-l CASP (Critical Assessment of Structure Prediction) experiments are held every two years. Recent rounds have seen dramatic increases in modeling accuracy, resulting from the introduction of deep learning methods: In 2018, for the first time, the folds of most proteins were correctly computed [1]; in 2020, the accuracy of many computed protein structures rivaled that of the corresponding experimental ones [2]; in 2022, there was an enormous increase in the accuracy of protein complexes [3]. We have seen the beginning of what deep learning methods may achieve in structural biology. In addition to further increases in the accuracy of protein complexes, methods are being developed for RNA structures, organic ligand-protein complexes, and for moving beyond single macromolecular structures to compute conformational ensembles. Accurate computational methods together with experimental data also offer the prospect of probing previously inaccessible biological systems. CASP has expanded its scope to provide critical assessment in all these areas. CASP is only possible with the generous participation of the experimental structural biology community in providing suitable targets: A total of over 1100 targets have been obtained over the previous CASP rounds. We are now requesting targets for the 2024 CASP16 experiment. We need challenge targets in the following areas: Single protein structures: The 2020 and 2022 CASPs showed that, so far, Alphafold2 and methods built around it are by far the most accurate [4]. But there are limitations, particularly for some proteins where only a shallow sequence alignment is available and for very large proteins (more than 1000 amino acids). The best results also require substantial amounts of computing resources, well beyond that of the AlphaFold2 default settings. Many new methods are continuing to appear and these may remove some of the remaining difficulties. All types of protein targets are needed, but especially those with shallow sequence alignments, without structural templates, and large proteins. Protein complexes: In the 2022 CASP15, advanced deep learning methods were applied to protein complexes for the first time [5]. The result was a huge improvement in accuracy compared with classical docking approaches. But overall, the results are still not at the level achieved for single proteins. So, in CASP16 we need all sorts of targets in this area so as to determine progress since then. We particularly need complexes where there is no evolutionary information across the protein-protein interfaces, for example, antibody-antigen complexes. (This CASP category is conducted in close collaboration with our colleagues at CAPRI - Critical Assessment of protein interactions [6]). Nucleic acid structures and complexes: In recognition of the major role nucleic acid structures and complexes play in biology, CASP now includes this class of target. A number of papers claiming successful RNA structure computation using deep learning methods have been published, but those participating in the 2022 CASP RNA category performed less well than classical approaches, and no methods were able to effectively address the two RNA protein-complexes included [7]. CASP needs a wide variety of RNA, DNA, and complexes as targets to see if this situation has changed. (This CASP category is conducted in close collaboration with RNApuzzles [8]). Organic ligand-protein complexes: This area is of major importance for computer-aided drug discovery. Earlier, there have been community experiments to assess the accuracy of methods, particularly SAMPL, CSAR, D3R, and a new one, CACHE, has recently started (http://cache-challenge.org ). These challenges have drawn strong international participation from researchers in both academia and industry. Here too, a number of promising deep learning papers have appeared, but in the 2022 CASP15 pilot, classical methods were still superior [9]. So, we need appropriate targets to see if progress has been made since. Ideally, these should be sets of three-dimensional protein-ligand complexes from drug discovery projects, but single targets would also be appreciated. Additionally, where available, we will assess non-structural quantities such as affinities or affinity rankings and other properties of pharmaceutical interest when these are available (small molecule pKs, and DMPK related properties). Ensembles of macromolecule conformations: It is now widely recognized that proteins and nucleic acids often adopt multiple conformations that can underpin their functions. In these cases, considering only a single protein or RNA conformation may be a significant oversimplification. The 2022 CASP15 included a pilot experiment to assess methods for computing multiple conformations, with encouraging results [10], but with limitations imposed by the available experimental data. For 2024, we seek not only cases of multiple experimental three-dimensional structures for the same macromolecule but also other types of data that might be used for assessment of computed conformation ensembles such as cryoEM, NMR, X-ray crystallography, SAXS, and/or cross-link data. Integrative modeling: The more powerful computational methods open up new possibilities for combination with sparse or low-resolution experimental data to investigate previously inaccessible biological structures and machines. CASP is interested in exploring these possibilities and so requests experimentally difficult targets where structure has nevertheless been obtained. In appropriate cases, we expect to be able to collaborate with other experimental groups to provide appropriate data from NMR, cross-linking or SAXS. There are three avenues to contribute a target to CASP: 1. (preferrable) Submit directly to CASP through our web-interface https://predictioncenter.org/casp16/targets_submission_form.cgi (requires a quick registration at https://predictioncenter.org/login.cgi if you do not have an account with us). 2. Email to targets@predictioncenter.org with your target suggestions or to discuss any questions. 3. Submit your structure to the PDB (on-hold) and designate it as a CASP target through PDB’s submission interface. The timeline for the 2024 CASP requires that targets are submitted starting now and until July 1. We would like to hear from you as soon as possible if you may have something suitable or have suggestions about other target sources. In order to maintain rigor, the experimental data for a target must not be publicly available until after computed structures have been collected. For assessment, CASP requires the experimental data by August 15, but the data can remain confidential after that. Target providers are invited to contribute to papers [11-15] for a special CASP issue of the journal Proteins. CASP organizers: John Moult, Krzysztof Fidelis, Andriy Kryshtafovych, Torsten Schwede, Maya Topf *References* 1. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XIII. Proteins 2019;87(12):1011-1020. 2. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XIV. Proteins 2021;89(12):1607-1617. 3. Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical assessment of methods of protein structure prediction (CASP)-Round XV. Proteins 2023;91(12):1539-1549. 4. Simpkin AJ, Mesdaghi S, Sanchez Rodriguez F, Elliott L, Murphy DL, Kryshtafovych A, Keegan RM, Rigden DJ. Tertiary structure assessment at CASP15. Proteins 2023;91(12):1616-1635. 5. Ozden B, Kryshtafovych A, Karaca E. The impact of AI-based modeling on the accuracy of protein assembly prediction: Insights from CASP15. Proteins 2023;91(12):1636-1657. 6. Lensink MF, Brysbaert G, Raouraoua N, Bates PA, Giulini M, Honorato RV, van Noort C, Teixeira JMC, Bonvin A, Kong R, Shi H, Lu X, Chang S, Liu J, Guo Z, Chen X, Morehead A, Roy RS, Wu T, Giri N, Quadir F, Chen C, Cheng J, Del Carpio CA, Ichiishi E, Rodriguez-Lumbreras LA, Fernandez-Recio J, Harmalkar A, Chu LS, Canner S, Smanta R, Gray JJ, Li H, Lin P, He J, Tao H, Huang SY, Roel-Touris J, Jimenez-Garcia B, Christoffer CW, Jain AJ, Kagaya Y, Kannan H, Nakamura T, Terashi G, Verburgt JC, Zhang Y, Zhang Z, Fujuta H, Sekijima M, Kihara D, Khan O, Kotelnikov S, Ghani U, Padhorny D, Beglov D, Vajda S, Kozakov D, Negi SS, Ricciardelli T, Barradas-Bautista D, Cao Z, Chawla M, Cavallo L, Oliva R, Yin R, Cheung M, Guest JD, Lee J, Pierce BG, Shor B, Cohen T, Halfon M, Schneidman-Duhovny D, Zhu S, Yin R, Sun Y, Shen Y, Maszota-Zieleniak M, Bojarski KK, Lubecka EA, Marcisz M, Danielsson A, Dziadek L, Gaardlos M, Gieldon A, Liwo A, Samsonov SA, Slusarz R, Zieba K, Sieradzan AK, Czaplewski C, Kobayashi S, Miyakawa Y, Kiyota Y, Takeda-Shitaka M, Olechnovic K, Valancauskas L, Dapkunas J, Venclovas C, Wallner B, Yang L, Hou C, He X, Guo S, Jiang S, Ma X, Duan R, Qui L, Xu X, Zou X, Velankar S, Wodak SJ. Impact of AlphaFold on structure prediction of protein complexes: The CASP15-CAPRI experiment. Proteins 2023;91(12):1658-1683. 7. Das R, Kretsch RC, Simpkin AJ, Mulvaney T, Pham P, Rangan R, Bu F, Keegan RM, Topf M, Rigden DJ, Miao Z, Westhof E. Assessment of three-dimensional RNA structure prediction in CASP15. Proteins 2023;91(12):1747-1770. 8. Magnus M, Antczak M, Zok T, Wiedemann J, Lukasiak P, Cao Y, Bujnicki JM, Westhof E, Szachniuk M, Miao Z. RNA-Puzzles toolkit: a computational resource of RNA 3D structure benchmark datasets, structure manipulation, and evaluation tools. Nucleic Acids Res 2020;48(2):576-588. 9. Robin X, Studer G, Durairaj J, Eberhardt J, Schwede T, Walters WP. Assessment of protein-ligand complexes in CASP15. Proteins 2023;91(12):1811-1821. 10. Kryshtafovych A, Montelione GT, Rigden DJ, Mesdaghi S, Karaca E, Moult J. Breaking the conformational ensemble barrier: Ensemble structure modeling challenges in CASP15. Proteins 2023;91(12):1903-1911. 11. Kretsch RC, Andersen ES, Bujnicki JM, Chiu W, Das R, Luo B, Masquida B, McRae EKS, Schroeder GM, Su Z, Wedekind JE, Xu L, Zhang K, Zheludev IN, Moult J, Kryshtafovych A. RNA target highlights in CASP15: Evaluation of predicted models by structure providers. Proteins 2023;91(12):1600-1615. 12. Alexander LT, Durairaj J, Kryshtafovych A, Abriata LA, Bayo Y, Bhabha G, Breyton C, Caulton SG, Chen J, Degroux S, Ekiert DC, Erlandsen BS, Freddolino PL, Gilzer D, Greening C, Grimes JM, Grinter R, Gurusaran M, Hartmann MD, Hitchman CJ, Keown JR, Kropp A, Kursula P, Lovering AL, Lemaitre B, Lia A, Liu S, Logotheti M, Lu S, Markusson S, Miller MD, Minasov G, Niemann HH, Opazo F, Phillips GN, Jr., Davies OR, Rommelaere S, Rosas-Lemus M, Roversi P, Satchell K, Smith N, Wilson MA, Wu KL, Xia X, Xiao H, Zhang W, Zhou ZH, Fidelis K, Topf M, Moult J, Schwede T. Protein target highlights in CASP15: Analysis of models by structure providers. Proteins 2023;91(12):1571-1599. 13. Alexander LT, Lepore R, Kryshtafovych A, Adamopoulos A, Alahuhta M, Arvin AM, Bomble YJ, Bottcher B, Breyton C, Chiarini V, Chinnam NB, Chiu W, Fidelis K, Grinter R, Gupta GD, Hartmann MD, Hayes CS, Heidebrecht T, Ilari A, Joachimiak A, Kim Y, Linares R, Lovering AL, Lunin VV, Lupas AN, Makbul C, Michalska K, Moult J, Mukherjee PK, Nutt WS, Oliver SL, Perrakis A, Stols L, Tainer JA, Topf M, Tsutakawa SE, Valdivia-Delgado M, Schwede T. Target highlights in CASP14: Analysis of models by structure providers. Proteins 2021;89(12):1647-1672. 14. Lepore R, Kryshtafovych A, Alahuhta M, Veraszto HA, Bomble YJ, Bufton JC, Bullock AN, Caba C, Cao H, Davies OR, Desfosses A, Dunne M, Fidelis K, Goulding CW, Gurusaran M, Gutsche I, Harding CJ, Hartmann MD, Hayes CS, Joachimiak A, Leiman PG, Loppnau P, Lovering AL, Lunin VV, Michalska K, Mir-Sanchis I, Mitra AK, Moult J, Phillips GN, Jr., Pinkas DM, Rice PA, Tong Y, Topf M, Walton JD, Schwede T. Target highlights in CASP13: Experimental target structures through the eyes of their authors. Proteins 2019;87(12):1037-1057. 15. Kryshtafovych A, Albrecht R, Basle A, Bule P, Caputo AT, Carvalho AL, Chao KL, Diskin R, Fidelis K, Fontes C, Fredslund F, Gilbert HJ, Goulding CW, Hartmann MD, Hayes CS, Herzberg O, Hill JC, Joachimiak A, Kohring GW, Koning RI, Lo Leggio L, Mangiagalli M, Michalska K, Moult J, Najmudin S, Nardini M, Nardone V, Ndeh D, Nguyen TH, Pintacuda G, Postel S, van Raaij MJ, Roversi P, Shimon A, Singh AK, Sundberg EJ, Tars K, Zitzmann N, Schwede T. Target highlights from the first post-PSI CASP experiment (CASP12, May-August 2016). Proteins 2018;86 Suppl 1(Suppl 1):27-50. -- YUHE LIANG, Ph.D. Biocuration Lead Deputy, RCSB Protein Data Bank Research Associate, Institute for Quantitative Biomedicine Rutgers, The State University of New Jersey 174 Frelinghuysen Road, Piscataway NJ 08554 P: 848.445.4938 | E: yuhe.liang@rcsb.org