Machine Learning Analysis
Supervised models on omics or clinical matrices produce calibrated risk estimates with nested cross-validation, serialized models, full code, and a Methods draft [1].
| Category | Details |
|---|---|
| Supported platforms / instruments | Feature matrices from bulk RNA-seq (DESeq2 VST or normalized counts), Olink NPX, MaxQuant or DIA-NN protein tables, metabolite peak tables, methylation beta values, clinical and laboratory covariates; vendor-normalized matrices accepted with documented upstream processing |
| Input requirements | Sample-level feature matrix (samples × features) plus outcome labels; survival endpoints require event indicator and follow-up time; feasibility reviewed with pmsampsize when binary or time-to-event; p >> n cohorts flagged before modeling [5, 6] |
| Reference builds supported | Human GRCh38 (GENCODE v44 / Ensembl 110); mouse GRCm39 (GENCODE vM33 / Ensembl 110); UniProt Swiss-Prot accessions for protein harmonization; HMDB 5.0 metabolite IDs when metabolomics features are included; not applicable for purely clinical tables |
| Primary tools (with versions) | scikit-learn 1.6.1; XGBoost 2.1.3; scikit-survival 0.24.1; glmnet 4.1-8 (R); SHAP 0.46.0; pmsampsize (R); limma 3.60; sva/ComBat when batch correction is scoped; clusterProfiler 4.20.0 for pathway context when scoped |
| Typical turnaround time | 3-5 weeks (single cohort, one outcome, ≤500 features, internal validation); 5-8 weeks (survival models, multi-cohort external validation). Confirmed at kickoff |
| Deliverable formats | .csv performance and feature tables; serialized models (.pkl, .joblib, or .rds); PDF/SVG ROC, calibration, and SHAP figures; DOME summary table; TRIPOD+AI checklist when clinical prediction reporting is scoped; commented R/Python scripts; Methods draft |
| Key cited best-practice reference | Walsh et al. [1], Nature Methods (DOME); Collins et al. [7], BMJ (TRIPOD+AI when scoped) |
| Custom / bespoke analysis | Non-standard outcomes, fusion models, client-specified metrics or thresholds, bespoke validation schemes, or domain-specific model families scoped at kickoff |
Predictive modeling fits supervised algorithms, including logistic regression, gradient-boosted trees, or survival forests, to estimate probabilities or risk scores for a pre-specified endpoint from high-dimensional biological or clinical predictors [3]. Biomarker discovery focuses on compact feature lists and selection stability. Predictive modeling asks whether those features generalize to unseen samples, with reported discrimination and calibration [1]. Unsupervised clustering does not train against labeled outcomes; these models do. A 2023 systematic review screened 682 PubMed articles (2017-2023) and critically reviewed 30 cancer ML biomarker studies, noting data-leakage risks and inconsistent evaluation among common limitations [4]. See the predictive modeling glossary.
Use predictive modeling when you have labeled outcomes and need out-of-sample performance estimates. A ranked feature list without outcome-model reporting, or a sample size too small for even regularized models, is a poor fit [5].
| Approach | Best for | Limitations | Approximate cost range |
|---|---|---|---|
| Predictive modeling (nested CV) | Outcome prediction, treatment-response stratification, survival risk scoring with calibration reporting | Needs labeled outcomes; small cohorts limit external validity; p >> n needs regularization and careful validation | Quote-based; moderate bioinformatics effort [1] |
| Biomarker discovery | Compact signatures, feature stability across resamples, exploratory panel construction | Does not by default deliver full outcome-model calibration or TRIPOD+AI reporting | Similar per-cohort cost; narrower deliverable scope |
| Classical multivariable regression (Cox, logistic) | Interpretable coefficients when predictors are pre-specified and low-dimensional | No nested hyperparameter tuning; overfitting risk when features >> samples without regularization | Lower when predictors are fixed; similar when high-dimensional |
Pepkio confirms the prediction target, endpoint type, primary metrics, and whether a holdout or external cohort exists [1, 7]. Sample-size rationale uses pmsampsize when binary or time-to-event endpoints apply [6].
Tools used: pmsampsize (R); project scoping templates
Output: Signed scope document with primary and secondary metrics
Sample IDs are matched between feature matrix and outcome table. Missingness, duplicates, class balance, and batch structure are audited.
Tools used: Custom Python/R validation scripts; pandas 2.2 / data.table
Output: sample_manifest.csv; feature_qc_summary.csv
Features are filtered by missing-value fraction and variance. Continuous predictors are scaled within cross-validation pipelines. Batch correction with limma or ComBat is applied within training folds only when batch is not confounded with the outcome [5].
Tools used: scikit-learn 1.6.1 ColumnTransformer; limma 3.60; sva/ComBat when scoped
Output: Analysis-ready feature matrix; preprocessing parameter log
Pepkio applies the seven guiding questions from Bernett et al. [9] before partitioning, checking whether preprocessing, feature selection, or repeated measures could leak test-fold information.
Tools used: Documented leakage checklist per Bernett et al. [9]
Output: leakage_audit.md
Outer cross-validation estimates generalization. An inner loop tunes hyperparameters and embedded feature selection [1]. Group-aware splitters keep repeated measures from the same patient in one fold.
Tools used: scikit-learn 1.6.1 StratifiedKFold, GridSearchCV; scikit-survival 0.24.1 for censored outcomes
Output: fold_assignments.csv (sample_id, outer_fold, inner_fold, split_role)
Pepkio compares elastic-net logistic regression, XGBoost, and survival forests against simpler baselines [3], with feature selection embedded inside each outer fold. Deep learning is scoped separately when justified by cohort size.
Tools used: scikit-learn 1.6.1; XGBoost 2.1.3; glmnet 4.1-8; scikit-survival 0.24.1
Output: Candidate model list with hyperparameter grids; serialized best models per fold
Discrimination (AUROC, AUPRC, or C-index) and calibration (reliability diagrams, Brier score, calibration slope) are reported from outer-fold predictions [7]. External performance is reported separately when a holdout or external cohort is available [5].
Tools used: scikit-learn 1.6.1 calibration_curve, brier_score_loss; scikit-survival 0.24.1 concordance_index_censored
Output: model_performance_summary.csv; calibration_metrics.csv
Pepkio documents threshold selection (Youden index on outer-fold predictions or a clinical cutoff) and reports sensitivity, specificity, and PPV at that threshold. Decision-curve analysis is included when scoped.
Tools used: scikit-learn 1.6.1 roc_curve; custom threshold summary scripts
Output: threshold_summary.csv (threshold, sensitivity, specificity, PPV, NPV)
SHAP values and coefficient tables identify top predictors. GO/KEGG annotation is included when gene or protein lists are in scope. Interpretability does not support causal claims [10].
Tools used: SHAP 0.46.0; clusterProfiler 4.20.0 when scoped
Output: SHAP summary plots; selected_features.csv; pathway enrichment tables
Pepkio assembles figures, serialized models, DOME summary table, TRIPOD+AI checklist when scoped, environment locks, README, and Methods draft.
Tools used: conda or renv lock files; DOME and TRIPOD+AI checklists
Output: Final deliverable bundle
There is no universal minimum. Riley et al. [6] and van Smeden et al. [11] show the 10 events-per-variable rule is unreliable as a sole criterion. Pepkio reviews event counts and predictor count with pmsampsize at kickoff and flags p >> n designs [5]. Feasibility is confirmed before modeling begins.
Within limits. Pepkio filters high-missingness features, documents exclusion counts, and may reduce dimensionality with embedded regularization inside nested CV. Severely underpowered cohorts or features with >50% missingness across samples are flagged before model training. Pepkio does not claim external validity without a holdout or external cohort [5].
Pepkio accepts sample-level matrices from bulk RNA-seq (DESeq2 VST or normalized counts), Olink NPX, MaxQuant or DIA-NN protein tables, metabolite peaks, methylation beta values, and clinical covariates. Vendor-normalized matrices require documented upstream processing and feature identifiers. Custom formats and MGI-derived matrices are scoped at kickoff.
Single-cohort projects with one binary outcome and ≤500 features often finish in 3-5 weeks from data receipt. Survival models, multi-cohort external validation, or bespoke validation schemes may need 5-8 weeks. Exact timelines are confirmed at kickoff with milestone check-ins after QC and before delivery.
Batch is diagnosed with PCA and correlation checks on the feature matrix. When batch is not confounded with the outcome, Pepkio applies limma removeBatchEffect or ComBat within training folds only [5]. Confounded batch-by-condition designs are flagged at kickoff. Correction proceeds only when a defensible contrast remains.
Yes. You retain full ownership of all code, scripts, and results. Pepkio delivers commented Python and/or R scripts with conda environment.yml or renv.lock files, organized by pipeline stage with a README. Models are serialized in standard .pkl, .joblib, or .rds formats. Delivery is via private Git repository or agreed file transfer.
Yes. Checkpoint reviews occur after input QC, after the leakage audit, and before final delivery. You can review outcome definitions, model family choices, threshold selection, and figure priorities within agreed scope. A dedicated scientific contact leads the project and incorporates feedback at each agreed milestone.
Support covers clarification of validation design, calibration methods, threshold rationale, and minor figure or table revisions within agreed scope (typically ≤20% of deliverables). Pepkio drafts Methods and Supplementary text for analyses we performed. Reviewer-requested re-analyses, new cohorts, or endpoints are scoped as separate milestones.
No. Pepkio is a fee-for-service provider and does not require co-authorship unless that is discussed in advance. Acknowledgment of bioinformatics support in the Acknowledgments section is the usual practice. Co-authorship is considered only when Pepkio scientists make substantial intellectual contributions beyond routine analysis.
Use classification for binary or multiclass outcomes at a fixed time point (for example, responder vs non-responder). Use survival models when follow-up is censored and time-to-event matters. Pepkio documents the endpoint in the scope document and reports C-index and Kaplan-Meier stratification for survival endpoints [7].
When a matched external cohort exists and is scoped at kickoff, Pepkio harmonizes feature identifiers (GENCODE v44 gene IDs, UniProt accessions) and reports external performance separately from nested-CV estimates [5]. Public-cohort validation is not included by default and depends on outcome overlap and feature overlap.
AUROC and AUPRC measure classification discrimination; C-index measures survival concordance. All are computed on outer-fold or holdout predictions, not training data [1]. Calibration uses Brier score, calibration slope, and reliability diagrams [7]. Confidence intervals use outer-fold or bootstrap resampling where sample size permits.
Tell us:
We'll tell you:
Contact us to start with a free consultation. Need everyday bench calculators? Try our free lab tools.
