Skip to content

Validation of Exomiser performance

Exomiser versions and validation method

Since the NGIS Jabbah release (September 2023) the Genomics England pipeline uses Exomiser version 13.2.0.

We validated the performance of the latest major release of Exomiser (v13) to identify known genetic diagnoses using 1869 variants in 1659 cases, representing all diagnostic variants reported in the outcome questionnaires in the Genomic Medicine Service by NHS GLHs at the time of analysis (Nov 2022).

The rankings of variants were compared to the previous version of Exomiser used in the pipeline (v12.0.0).

Note

Validations were performed using Exomiser (v13.1.0). There are no expected deviations in performance from the minor updated version of the software integrated into the pipeline (v13.2.0)

Sensitivity for known diagnostic variants

The 1869 variant(s) reported as diagnostic were returned in the top 3 ranked candidates from Exomiser (v13) for 1709/1869 variants (91%, 95% CI 90%-93%). This favourably compares with 80% (95% CI 77%-81%) in the top 3 using the previous version of Exomiser (v12.0.0).

Differences in behaviour between versions

Despite the overall improved performance, 68 variants that were previously prioritised rank 1-3 (using Exomiser v12) have dropped below rank 3 using Exomiser v13.

We identified several common reasons for why these variants were below rank 3 in Exomiser 13.1.0:

  • Low phenotype match which contributes to a low score (25%)
  • The score of a diagnostic variant stayed the same but a different variant improved in score (16%)
  • MAF >1%, which while below the threshold of 2% still contributes to a low score (15%)
  • The variant was still ranked highly despite being outside of rank 3 (15% in rank 4-5).

Overall, including ranks 1-5 instead of ranks 1-3 increases the recall of variants from 0.91 to 0.94 (95% CI 0.93- 0.95).

Limitations

One limitation of Exomiser is that heteroplasmy is not taken into account for mitochondrial variants. We see that Exomiser prioritises mitochondrial variants that would usually be filtered by the thresholds used for heteroplasmic variants in the rare disease tiering algorithm (allele fraction ≥ 0.05). As a result, care should be taken interpreting these variants and the heteroplasmy level should be checked.