Benchmarks
Qualitiative Detection
Benchmarks have been produced adopting the FLUX SIMULATOR to simulate different experimental conditions. Subsequently, the FLUX CAPACITOR has been evaluated in its ability to re-assign the reads of a locus to single transcripts overlapping there. As a reference transcriptome, the currently 65,803 "known Gene" transcripts for the human genome are recruited. The FLUX SIMULATOR created a scenario in which 10,372 of the expressed spliceforms have been detected in an RNAseq experiment. In total, the loci with detected reads comprised 33,676 different transcripts. Furthermore, the expressed loci harboured 20,210 different AS events, of which 11,528 are expressed in both variants.
| | TP | FN | TN | FP | sensitivity | specificity |
Transcripts FLUX CAPACITOR EXON-COUNT | 8,421 5,743 | 1,951 4,629 | 22,413 20,395 | 891 2,909 | 81.19% 55.37% | 96.18% 87.52% |
AS events FLUX CAPACITOR EXON-COUNT | 7,724 6,495 | 3,804 5,033 | 7,547 6,957 | 1,135 1,725 | 67.00% 56.34% | 86.93% 80.13% |
Table - Qualitative benchmark of abundancy predictions. Results yielded by the signal separation of the FLUX CAPACITOR have been benchmarked against the currently applied technique of counting reads along exons, taking into account biases by overlapping exonic stretches. Transcripts have been identified as "expressed" if there has been a positive detection of all of their exons, expressed AS events consequently required a positive detection along both variants. Clearly, counting reads along exons misses many events where the supporting spliceforms did not produce any reads in the region of the event, which primarily manifests in a low sensitivity.
Quantitative Prediction
Predictions from the FLUX CAPACITOR are related in their quantitative correctness, again in comparison to the results achieved by counting reads along exonic stretches. As a measure, the relative error to the originally simulated coverage of the respective transcript or spliceform has been measured. False positive predictions are assigned an relative error of 100%, all other predictions are assigned the relative error of
| covpred - covref | / covref
where
covpred is the predicted coverage (measured usually in rpkm = reads per kilobase per million mapped reads
covref is the reference coverage, as obtained by knowing the anterior simulation step

Figure - Quantitative error in abundancy predictions. Predictions from the FLUX CAPACITOR again are compared to results yielded by counting reads that fall into the region of exonic stretches. Predictions of transcripts have been separated in different categories, according to their true abundany. According to the plot the error of wrongly predicted transcript abundancies is systematical and independent of the expression. Flow networks as adopted by the FLUX CAPACITOR are able to separate the signal in a subtile way and hence perform better -- achieving convergence as informational readcount increases. Obviously, the combined signal along variants of AS events stabilizes biases in both techniques and results in <50% error for over 75% of the events predictions for the FLUX CAPACITOR.