Gonzalez-Porta, M. | bioRxiv Science

bioRxiv2023

Development and validation of a pharmacogenomics reporting workflow based on the Illumina Global Screening Array chip

BackgroundMicroarrays are a well-established and widely adopted technology capable of interrogating hundreds of thousands of loci across the human genome. Combined with imputation to cover common variants not included in the chip design, they offer a cost-effective solution for large-scale genetic studies. Beyond research applications, this technology can be applied for testing pharmacogenomics, nutrigenetics, and complex disease risk prediction. However, establishing clinical reporting workflows requires a thorough evaluation of the assays performance, which is achieved through validation studies. In this study, we performed pre-clinical validation of a genetic testing workflow based on the Illumina Global Screening Array for 25 pharmacogenomic-related genes. MethodsTo evaluate the accuracy of our workflow, we conducted multiple pre-clinical validation studies. Here, we present the results of accuracy and precision assessments, involving a total of 73 cell lines. These assessments encompass reference materials from the Genome-In-A-Bottle (GIAB), the Genetic Testing Reference Material Coordination Program (GeT-RM) projects, as well as additional samples from the 1000 Genomes project (1KGP). We conducted an accuracy assessment of genotype calls for target loci in each indication against established truth sets. ResultsIn our per-sample analysis, we observed a mean analytical sensitivity of 99.39% and specificity 99.98%. We further assessed the accuracy of star-allele calls by relying on established diplotypes in the GeT-RM catalogue or calls made based on 1KGP genotyping. On average, we detected a diplotype concordance rate of 96.47% across 14 pharmacogenomic-related genes with star allele-calls. Lastly, we evaluated the reproducibility of our findings across replicates and observed 99.48% diplotype and 100 % phenotype inter-run concordance. ConclusionOur comprehensive validation study demonstrates the robustness and reliability of the developed workflow, supporting its readiness for further development for applied testing.

genetics↗

bioRxiv2018

Best Practices for Benchmarking Germline Small Variant Calls in Human Genomes

Assessing accuracy of NGS variant calling is immensely facilitated by a robust benchmarking strategy and tools to carry it out in a standard way. Benchmarking variant calls requires careful attention to definitions of performance metrics, sophisticated comparison approaches, and stratification by variant type and genome context. The Global Alliance for Genomics and Health (GA4GH) Benchmarking Team has developed standardized performance metrics and tools for benchmarking germline small variant calls. This team includes representatives from sequencing technology developers, government agencies, academic bioinformatics researchers, clinical laboratories, and commercial technology and bioinformatics developers for whom benchmarking variant calls is essential to their work. Benchmarking variant calls is a challenging problem for many reasons:\n\nO_LIEvaluating variant calls requires complex matching algorithms and standardized counting because the same variant may be represented differently in truth and query callsets.\nC_LIO_LIDefining and interpreting resulting metrics such as precision (aka positive predictive value = TP/(TP+FP)) and recall (aka sensitivity = TP/(TP+FN)) requires standardization to draw robust conclusions about comparative performance for different variant calling methods.\nC_LIO_LIPerformance of NGS methods can vary depending on variant types and genome context; and as a result understanding performance requires meaningful stratification.\nC_LIO_LIHigh-confidence variant calls and regions that can be used as \"truth\" to accurately identify false positives and negatives are difficult to define, and reliable calls for the most challenging regions and variants remain out of reach.\nC_LI\n\nWe have made significant progress on standardizing comparison methods, metric definitions and reporting, as well as developing and using truth sets. Our methods are publicly available on GitHub (https://github.com/ga4gh/benchmarking-tools) and in a web-based app on precisionFDA, which allow users to compare their variant calls against truth sets and to obtain a standardized report on their variant calling performance. Our methods have been piloted in the precisionFDA variant calling challenges to identify the best-in-class variant calling methods within high-confidence regions. Finally, we recommend a set of best practices for using our tools and critically evaluating the results.

genomics↗