Clinic 17

Kernel Choice

Compare linear and nonlinear models under the split that matches deployment, including their fitting cost.

Back To Clinics Open SVM Topic Open Honest Splits

Situation

A clinical-record classification exercise has multiple rows per patient, and deployment concerns new patients. The first comparison used a random row split. You repeat development with patients kept disjoint between training and validation. This is a modeling exercise, not a clinical deployment recommendation.

Artifact Packet

The scores and timings are illustrative. Each column uses a consistent protocol across models, but the random and grouped validation populations differ. No repeat-run or paired uncertainty estimates are supplied.

model	random-split AUC	new-patient AUC	fit seconds
linear	0.812	0.785	8
rbf	0.834	0.791	150
polynomial	0.821	0.778	240
domain_kernel	0.818	0.783	20

Decision Prompt

1. Which validation column matches previously unseen patients?
2. Is the RBF improvement worth its measured cost?
3. What uncertainty estimate would support that comparison?
4. Where do scaling and kernel hyperparameter tuning belong?

Strong Reasoning Looks Like

• choose the patient-disjoint evaluation for the stated deployment
• fit scaling on training folds and tune C, kernel parameters, and other choices within the training data
• compare paired prediction errors on common validation cases, respecting patient groups
• report latency, fit cost, and the uncertainty in the improvement

Run The Clinic In Browser

The runner prints this fixed illustrative packet and recalculates any derived columns. It does not run a new training experiment. Edit its PACKET values to explore the decision.

Reference Reveal

Open after writing your note

Use **the linear model as a cheap baseline and RBF as the accuracy challenger**. RBF's grouped AUC is 0.006 higher and its measured fitting time is substantially longer. The packet alone does not decide whether that tradeoff is valuable or whether the gain is repeatable. State the acceptable gain and cost, then collect the needed evidence. Either choice can be defended conditionally. The first repair is the deployment-matched split. Scaling affects distances and therefore the meaning of RBF gamma; omitting scaling is not, by itself, data leakage. Fitting a scaler or selecting hyperparameters using validation information can contaminate evaluation. A fold-score SD for one model is not an uncertainty interval for the difference between two models. Use paired comparisons and an uncertainty method that respects the grouping and evaluation design. A domain-designed kernel must also satisfy the mathematical requirements of the chosen kernel method; its name does not guarantee useful structure.

What To Do Next

1. open SVM Margins and Kernels for the knob-by-knob tuning logic
2. open Honest Splits and Baselines for the grouped-split recipe
3. open Leakage Patterns for the scaler-leak pattern this clinic relies on
4. rerun your own leaderboard under the correct group-aware split; if your top-of-board kernel survives, the win is real