Configure Your Split
How split sizes are calculated
- Percentage mode: count = total × (ratio ÷ 100) for each split.
- Balanced rounding: floors each split, then distributes remaining records to the largest fractional parts.
- Validation optional: when disabled, validation ratio becomes 0%.
- Normalization: if ratios don’t sum to 100%, optional scaling keeps proportions consistent.
- Counts mode: entered counts are used; remainder can be assigned or left unassigned.
Steps for reliable dataset splitting
- Enter the total number of records in your dataset.
- Select percentage or count mode based on your workflow.
- Choose whether you need a validation set.
- Pick a rounding method; use Balanced for exact totals.
- Enable shuffling and set a seed for reproducibility.
- Optionally paste class counts and enable stratified planning.
- Press Calculate Split and export CSV/PDF for documentation.
Sample engineering dataset (first rows)
| Record ID | Sensor (g) | Temperature (°C) | RPM | Status |
|---|---|---|---|---|
| ENG-0001 | 0.12 | 34.6 | 1450 | Normal |
| ENG-0002 | 0.18 | 35.1 | 1462 | Normal |
| ENG-0003 | 0.62 | 38.9 | 1510 | Warning |
| ENG-0004 | 1.24 | 41.7 | 1564 | Fault |
| ENG-0005 | 0.16 | 34.9 | 1456 | Normal |
This calculator sizes splits; your pipeline performs the actual row selection.
Selecting ratios for engineering datasets
Engineering models often perform best when training receives 60–85% of records, leaving enough data for unbiased evaluation. For small datasets under 1,000 samples, prefer 80/10/10 or cross‑validation to reduce variance. For large sensor logs above 100,000 rows, 70/15/15 provides stable metrics while keeping validation large enough for tuning. Choose a test set that mirrors deployment conditions, not laboratory-only data. Always report the chosen ratio.
Preventing leakage in time-based measurements
Many engineering datasets are temporal: vibration streams, SCADA readings, weathered material tests, or production runs. Random shuffling can leak future information into training when adjacent timestamps share patterns. Use chronological splitting: train on earlier periods, validate on mid periods, test on the most recent window. If equipment upgrades occurred, split by era to reflect drift. When data is grouped by asset, split by machine ID, not by row. This improves field reliability.
Stratified planning for rare failure modes
Failure prediction and defect detection face class imbalance, where failures may be 0.1–5% of samples. A naive split can place too few failures in validation or test, making metrics unstable. Stratified planning allocates each class proportionally to train, validation, and test counts. If you have 200 failures and use 70/15/15, aim for about 140/30/30 failures per split. For rare modes, increase test size or pool periods.
Rounding and reproducibility in pipelines
Operational pipelines require split sizes that sum to the dataset total, especially when records are indexed in databases or files. Balanced rounding floors each split then assigns leftover records to the largest fractional remainders, preserving ratios while maintaining totals. Fixed random seeds make shuffling deterministic, allowing teams to reproduce experiments and compare models fairly across teams. When ratios do not add to 100%, normalization rescales them before computing counts, preventing silent under-allocation.
Documenting splits for audits and reviews
In engineering programs, datasets are often shared across design, testing, and safety groups, so split documentation matters. Record the total size, chosen ratios, rounding method, shuffle seed, and any grouping rule used. Exporting results to CSV supports traceability in model cards and validation reports. A lightweight PDF snapshot helps during reviews, supplier audits, or regulatory submissions. Recompute splits whenever new batches, sensors, or labeling criteria are introduced. This keeps comparisons accurate always.
Questions about dataset splitting
What split ratio should I use for most engineering ML projects?
Start with 70/15/15 for large datasets. Use 80/10/10 for small datasets under ~1,000 records. If data is expensive to label, consider k-fold validation and keep the final test set untouched.
Why does my total not match after rounding?
Simple rounding can add or drop records. Balanced rounding floors each split then assigns leftover records based on fractional parts, ensuring train+validation+test equals the total.
When should I turn off the validation set?
Turn it off when you will tune using cross‑validation, when you already have a fixed development set, or when the dataset is tiny and you need more training data. Keep a separate test set for final reporting.
How should I split time-series or production-line data?
Avoid random shuffling across time. Split chronologically or by production batch, and keep entire machines/assets in one split to prevent leakage from correlated samples.
What is stratified planning and when is it helpful?
Stratified planning targets similar class proportions in each split. It’s useful for rare failure modes and imbalanced defect labels, where a random split might leave too few positives in validation or test.
What does the shuffle seed do?
A seed makes the shuffling order reproducible. Using the same seed and options lets different team members regenerate identical splits, supporting fair model comparisons and auditability.