CAPRI: Critical Assessment of PRediction of Interactions
Community wide experiment on the comparative evaluation of protein-protein docking for structure prediction
Hosted By EMBL/EBI-MSD Group
Schedule of Round 26 extension
Monday Feb. 20 - Round extension opens
Download additional information - 20th Feb 2012
Sunday March 18 Deadline for submission.
Round 26
Round 26 ID mapping from group number to Accessor Number for Target 53
Round 26 ID mapping from group number to Accessor Number for Target 54
NOTE The Coordinates are not yet available for download
T53 result summary
T54 result summary
Results for targets 55 and 56
General Summary
T55 result summary for Round 1
T55 result summary for Round 2
T56 result summary for Round 1
T56 result summary for Round 2
Target 53
Results T53 - click here to see the results
Full results (regular assessment):
Clash threshold | 62.31 |
average | 18.21 |
std dev | 22.05 |
| Predictor | Uploader | Scorer |
Nr groups | 42 | 15 | 13 |
High Accuracy (***) | 1 ( 1) | 0 ( 0) | 0 ( 0 <- 0) |
Medium (**) | 11 ( 9) | 9 ( 5) | 12 ( 9 <- 4) |
Acceptable (*) | 31 (17) | 92 (12) | 24 ( 8 <- 4) |
Incorrect | 321 (39) | 1090 (14) | 85 (13 <- 12) |
Clashes | 20 ( 5) | 209 ( 4) | 9 ( 3 <- 2) |
Low ID | 0 ( 0) | 0 ( 0) | 0 ( 0 <- 0) |
Total | 384 (40) | 1400 (14) | 130 (13 <- 13) |
Numbers in parentheses refer to the number of different source groups producing these models
(ie z scorer groups recognize y out of x uploader groups)
Target 54
Results T54 - click here to see the results
Full results (regular assessment):
Clash threshold | 64.14 |
average | 17.43 |
std dev | 23.36 |
| Predictor | Uploader | Scorer |
Nr groups | 41 | 16 | 13 |
High Accuracy (***) | 0 ( 0) | 0 ( 0) | 0 ( 0 <- 0) |
Medium (**) | 0 ( 0) | 1 ( 1) | 0 ( 0 <- 0) |
Acceptable (*) | 6 ( 4) | 18 ( 7) | 0 ( 0 <- 0) |
Incorrect | 350 (39) | 1196 (14) | 119 (13 <- 14) |
Clashes | 21 ( 6) | 185 ( 4) | 11 ( 3 <- 2) |
Low ID | 0 ( 0) | 0 ( 0) | 0 ( 0 <- 0) |
Total | 377 (39) | 1400 (14) | 130 (13 <- 14) |
Numbers in parentheses refer to the number of different source groups producing these models
(ie z scorer groups recognize y out of x uploader groups)
Round 26 - Results for targets 55 and 56
All metrics are calculated across all positions, as well as for just the mutations at interface positions
(T55: 33, 46, 47, 49, 50, 53, 54, 56, 57, 60, 61, 64, 65, 66, 69; T56: 12, 13, 14, 15, 17, 18, 20, 21, 24, 25, 27, 36, 39, 40, 42 ).
The first round metrics were calculated across all provided prediction for non-"wild type" mutations. The second round metrics were
calculated across all the provided predictions which were not in the released data set. The number of predictions considered for each
group is specified by the Nmut columns.
The Kendall values are the all-against-all Kendall tau rank order correlation for the continuous value predictions against the experimental
log2(enrichment ratio) values. To allow for metrics where predicted values may not be comparable across different positions, the Per Res Gamma metric
is a derivative of Goodman and Kruskal's gamma, where the counted concordant and discordant pairs are formed solely between mutations at a single position.
Frac correct is the fraction of correct class predictions, using experimental log2(enrichment ratio) cutoffs of +2.0 and -2.0 for determining the beneficial/neutral/deleterious class specification.
Download results for targets 55 and 56
References:
- Optimization of affinity, specificity, and function of designed Influenza inhibitors using next generation sequencing
Whitehead TA, Chevalier A, Song Y, Dreyfus C, Fleishman SJ, De Mattos C, Myers CA, Kamisetty H, Blair P, Wilson IA, Baker D.
Nature Biotechnology vol. 30, June 2012
Download complete experimental data set
- Next-generation protein engineering targets influenza New and Views
SJ Wodak
Nature Biotechnology vol. 30, June 2012
Target 55 Round 1
Group | Wtseq | Nmut | Nmut (Int.) | Per Res Gamma | Per Res Gamma (Int.) | Kendall | Kendall (Int.) | Frac correct | Frac correct (Int.) |
Mitsuo Iwadate | exp | 18 | 4 | 0.000 | -1.000 | 0.105 | 0.333 | 0.444 | 0.50 |
Julie Mitchell | *PDB* | 1007 | 285 | 0.072 | 0.084 | 0.061 | 0.159 | 0.568 | 0.71 |
Chaok Seok | exp | 1007 | 285 | 0.043 | 0.097 | 0.080 | 0.142 | 0.461 | 0.55 |
Martin Zacharias | *PDB* | 1007 | 285 | 0.016 | 0.102 | 0.098 | 0.216 | 0.473 | 0.37 |
Paul Bates | *PDB* | 1007 | 285 | -0.004 | 0.004 | 0.094 | 0.157 | 0.504 | 0.57 |
Haruki Nakamura | exp | 1007 | 285 | 0.013 | 0.010 | 0.141 | 0.223 | 0.484 | 0.43 |
Sergei Grudinin | *PDB* | 1007 | 285 | 0.012 | 0.107 | 0.077 | 0.089 | 0.461 | 0.47 |
SurFit,Daron Standley | exp | 1007 | 285 | 0.030 | 0.007 | 0.066 | 0.084 | 0.502 | 0.49 |
Xiaoqin Zou | exp | 1007 | 285 | 0.001 | 0.130 | 0.163 | 0.247 | 0.341 | 0.49 |
Zhiping Weng | *PDB* | 1007 | 285 | 0.119 | 0.084 | 0.224 | 0.217 | 0.557 | 0.55 |
Jeffrey J. Gray | *PDB* | 1007 | 285 | 0.031 | 0.028 | 0.166 | 0.188 | 0.539 | 0.52 |
Dmitry Korkin | exp | 1007 | 285 | 0.009 | -0.063 | 0.039 | 0.012 | 0.439 | 0.28 |
Alexandre M.J.J. Bonvin | exp | 988 | 266 | 0.013 | 0.030 | 0.088 | 0.200 | 0.498 | 0.47 |
Juan Fernandez-Recio | *PDB* | 1007 | 285 | 0.180 | 0.077 | 0.295 | 0.278 | 0.628 | 0.66 |
Yves Dehouck | *PDB* | 1007 | 285 | 0.151 | 0.130 | 0.286 | 0.252 | 0.582 | 0.59 |
Huan-Xiang Zhou | exp | 1007 | 285 | 0.074 | 0.033 | 0.165 | 0.236 | 0.504 | 0.52 |
Mayuko Takeda-Shitaka | *PDB* | 1007 | 285 | 0.069 | 0.172 | 0.123 | 0.152 | 0.478 | 0.55 |
Daisuke Kihara | *PDB* | 1007 | 285 | -0.022 | -0.002 | 0.054 | 0.179 | 0.259 | 0.51 |
Yi Xiao | exp | 1007 | 285 | 0.128 | 0.103 | 0.131 | 0.111 | 0.500 | 0.49 |
Sandor Vajda | *PDB* | 164 | 163 | 0.210 | 0.210 | 0.134 | 0.133 | 0.677 | 0.67 |
Meng Cui | *PDB* | 1007 | 285 | 0.013 | 0.040 | 0.068 | 0.166 | 0.417 | 0.49 |
Samuel Flores | *PDB* | 601 | 260 | 0.112 | 0.151 | 0.232 | 0.258 | 0.448 | 0.58 |
Target 55 Round 2
Group | Wtseq | Nmut | Nmut (Int.) | Per Res Gamma | Per Res Gamma (Int.) | Kendall | Kendall (Int.) | Frac correct | Frac correct (Int.) |
Julie Mitchell | exp | 530 | 150 | 0.097 | 0.150 | 0.263 | 0.298 | 0.532 | 0.65 |
Chaok Seok | exp | 530 | 150 | 0.083 | 0.183 | 0.088 | 0.103 | 0.440 | 0.51 |
Paul Bates | *PDB* | 530 | 150 | 0.303 | 0.324 | 0.514 | 0.400 | 0.547 | 0.69 |
Haruki Nakamura | exp | 530 | 150 | 0.039 | 0.115 | 0.097 | 0.217 | 0.443 | 0.39 |
Sergei Grudinin | *PDB* | 530 | 150 | -0.021 | 0.088 | 0.041 | 0.048 | 0.455 | 0.46 |
Zhiping Weng | exp | 530 | 150 | 0.130 | 0.110 | 0.335 | 0.249 | 0.643 | 0.66 |
Dmitry Korkin | *PDB* | 530 | 150 | 0.002 | -0.018 | 0.038 | 0.018 | 0.404 | 0.31 |
Alexandre M.J.J. Bonvin | exp | 530 | 150 | 0.128 | 0.168 | 0.157 | 0.119 | 0.502 | 0.53 |
Juan Fernandez-Recio | exp | 530 | 150 | 0.261 | 0.239 | 0.495 | 0.403 | 0.696 | 0.79 |
Mayuko Takeda-Shitaka | NA | 490 | 134 | 0.211 | 0.225 | 0.381 | 0.310 | 0.600 | 0.75 |
Daisuke Kihara | *PDB* | 530 | 150 | -0.045 | -0.052 | 0.032 | 0.158 | 0.255 | 0.48 |
Yi Xiao | exp | 530 | 150 | 0.090 | 0.089 | 0.199 | 0.253 | 0.532 | 0.49 |
Zhiping Weng | 855 | 285 | 0.053 | 0.076 | 0.227 | 0.254 | 0.468 | 0.54 |
Target 56 Round 1
Group | Wtseq | Nmut | Nmut (Int.) | Per Res Gamma | Per Res Gamma (Int.) | Kendall | Kendall (Int.) | Frac correct | Frac correct (Int.) |
Mitsuo Iwadate | exp | 332 | 114 | -0.030 | -0.052 | 0.019 | -0.015 | 0.654 | 0.72 |
Julie Mitchell | *PDB* | 855 | 285 | 0.053 | 0.022 | 0.056 | 0.092 | 0.572 | 0.61 |
Chaok Seok | exp | 855 | 285 | 0.110 | 0.179 | 0.035 | 0.133 | 0.427 | 0.52 |
Martin Zacharias | *PDB* | 855 | 285 | 0.119 | 0.176 | 0.029 | 0.132 | 0.382 | 0.44 |
Paul Bates | *PDB* | 853 | 284 | 0.028 | 0.057 | 0.085 | 0.200 | 0.402 | 0.49 |
Haruki Nakamura | exp | 855 | 285 | 0.091 | 0.102 | 0.079 | 0.092 | 0.357 | 0.35 |
Sergei Grudinin | *PDB* | 855 | 285 | 0.052 | 0.076 | 0.041 | 0.174 | 0.435 | 0.55 |
SurFit,Daron Standley/td> | exp | 855 | 285 | 0.041 | 0.060 | 0.129 | 0.248 | 0.432 | 0.49 |
Xiaoqin Zou | exp | 855 | 285 | 0.077 | 0.064 | 0.044 | 0.054 | 0.586 | 0.52 |
Zhiping Weng | *PDB* | 855 | 285 | 0.085 | 0.125 | 0.214 | 0.310 | 0.566 | 0.57 |
Jeffrey J. Gray | *PDB* | 822 | 285 | 0.097 | 0.119 | 0.139 | 0.223 | 0.482 | 0.47 |
Dmitry Korkin | exp | 855 | 285 | 0.035 | 0.018 | 0.077 | 0.070 | 0.357 | 0.34 |
Alexandre M.J.J. Bonvin | exp | 855 | 285 | -0.015 | 0.030 | 0.016 | 0.124 | 0.337 | 0.35 |
Juan Fernandez-Recio | *PDB* | 855 | 285 | 0.099 | 0.147 | 0.172 | 0.289 | 0.532 | 0.58 |
Huan-Xiang Zhou | exp | 855 | 285 | 0.075 | 0.048 | 0.147 | 0.161 | 0.421 | 0.49 |
Mayuko Takeda-Shitaka | *PDB* | 855 | 285 | 0.023 | 0.021 | 0.054 | 0.126 | 0.488 | 0.53 |
Daisuke Kihara | *PDB* | 855 | 285 | -0.001 | 0.070 | 0.015 | 0.098 | 0.290 | 0.41 |
Yi Xiao | exp | 836 | 285 | -0.168 | -0.494 | -0.029 | -0.002 | 0.287 | 0.27 |
Sandor Vajda | *PDB* | 200 | 160 | 0.207 | 0.144 | 0.173 | 0.141 | 0.665 | 0.63 |
Meng Cui | *PDB* | 854 | 284 | -0.016 | -0.047 | 0.047 | 0.140 | 0.370 | 0.43 |
Target 56 Round 2
Group | Wtseq | Nmut | Nmut (Int.) | Per Res Gamma | Per Res Gamma (Int.) | Kendall | Kendall (Int.) | Frac correct | Frac correct (Int.) |
Julie Mitchell | exp | 450 | 150 | -0.028 | 0.170 | 0.191 | 0.302 | 0.638 | 0.71 |
Chaok Seok | exp | 450 | 150 | 0.075 | 0.205 | 0.061 | 0.153 | 0.451 | 0.55 |
Paul Bates | *PDB* | 447 | 148 | 0.148 | 0.132 | 0.406 | 0.458 | 0.633 | 0.69 |
Sergei Grudinin | *PDB* | 450 | 150 | 0.011 | -0.007 | 0.029 | 0.142 | 0.451 | 0.53 |
Xiaoqin Zou | exp | 450 | 150 | 0.028 | -0.007 | 0.116 | 0.049 | 0.592 | 0.55 |
Zhiping Weng | exp | 450 | 150 | 0.044 | 0.083 | 0.209 | 0.321 | 0.580 | 0.65 |
Dmitry Korkin | *PDB* | 450 | 150 | 0.041 | 0.079 | 0.045 | -0.124 | 0.364 | 0.29 |
Alexandre M.J.J. Bonvin | exp | 450 | 150 | 0.087 | 0.262 | 0.130 | 0.242 | 0.509 | 0.59 |
Juan Fernandez-Recio | exp | 450 | 150 | 0.196 | 0.247 | 0.418 | 0.447 | 0.747 | 0.78 |
Yves Dehouck | exp | 450 | 150 | 0.131 | 0.163 | 0.393 | 0.438 | 0.689 | 0.71 |
Mayuko Takeda-Shitaka | NA | 325 | 91 | 0.188 | 0.220 | 0.278 | 0.239 | 0.702 | 0.76 |
ROUND 26 Timeline
2nd Nov. 2011 - Registration opens for Round 26
7th Nov. 2011 - Round 26 opens - Prediction of targets 53,54 and Scoring for targets 55,56
20th Nov. 2011 - Deadline for submitting models for target 53, 54
24th Nov 2011 - Scoring starts for targets 53, 54
27th Nov. 2011 - Deadline for scoring submission for targets 53, 54
22nd Dec. 2011 - Deadline for scoring submission for targets 55, 56
Target T53 a protein-protein complex (homology/unbound)
Target T54 a protein-protein complex (homology/unbound)
Target T55 (scoring only) is a large set (>500) of point mutations made in a protein engineered to bind another; given the X-ray structure of one complex, the challenge is to predict the effect of each mutation on the interaction.
Target T56 (scoring only) is a large set (>500) of point mutations made in a protein engineered to bind another; given the X-ray structure of one complex, the challenge is to predict the effect of each mutation on the interaction.
Documentation and Targets Round 26
Participants
Management Group
Prediction formats
Download Opens 7th Nov 2011
Upload Opens 7th Nov 2011
Scoring download open 24th Nov 2011
Entries
Current submissions
SCORERS
Scorers can download the server set of submissions from
http://www.scmbb.ulb.ac.be/capri/
using the same login user/pass as distributed for Capri round26. Full details
and deadlines are given on this site.
Email Problems or Queries to Sameer Velankar
![spacer](http://www.ebi.ac.uk/inc/images/spacer.gif) |