Data Loader Best Practices: Batch Sizes, Bulk API, and Surviving Large Imports
Data Loader is deceptively simple. The UI makes it look like you just pick a CSV and click Import. But the settings you choose determine whether your import takes 5 minutes or 5 hours, and whether it triggers every automation in your org or slips through cleanly. The three settings that matter most are batch size, API mode (Bulk vs SOAP), and the "Use Bulk API" checkbox.
SOAP API (the default) processes records in batches of 200. Each batch is a separate API call and each record in the batch triggers all automations (flows, triggers, validation rules). For imports under 10,000 records, this is fine. For anything larger, switch to Bulk API. Bulk API sends records in batches of up to 10,000, processes them asynchronously, and uses fewer API calls. For massive imports (100K+), Bulk API with serial mode (not parallel) prevents lock contention on parent records.
Always do a dry run first. Export the records you plan to import, run them through a sandbox with the same automations as production, and check the results. Disable unnecessary automations before large imports: deactivate flows, set custom metadata toggles to skip triggers, and temporarily adjust validation rules. Then re-enable everything after. This is not cutting corners — it is how every experienced admin handles large data operations.
Code Example
// Data Loader Settings Cheat Sheet:
// Small imports (< 10K records):
// Settings → Batch Size: 200 (default)
// Use Bulk API: unchecked
// Insert Null Values: check only if you WANT to blank fields
// Medium imports (10K - 100K records):
// Settings → Batch Size: 2000
// Use Bulk API: checked
// Serial Mode: unchecked (parallel is faster)
// Large imports (100K+ records):
// Settings → Batch Size: 5000
// Use Bulk API: checked
// Serial Mode: CHECKED (prevents lock errors)
// Enable Bulk API Zip: checked (faster network transfer)
// Automation management for large imports:
// 1. Custom Metadata toggle approach:
// Import_Settings__mdt.Skip_Triggers__c = true
// (Check this in your trigger handler first line)
// 2. In Apex trigger handler:
public class TriggerConfig {
public static Boolean skipTriggers {
get {
if (skipTriggers == null) {
Import_Settings__mdt config =
Import_Settings__mdt.getInstance('Default');
skipTriggers = config?.Skip_Triggers__c ?? false;
}
return skipTriggers;
}
set;
}
}
// In trigger:
if (TriggerConfig.skipTriggers) return;
// 3. Deactivate flows via Setup before import
// 4. Re-enable everything after import completesNeed this implemented in your org?
I've shipped these patterns in production for 10+ years.
View Consulting →Enjoyed this? Get more like it.
Glen's Musings — AI, investing, and building things. Occasional. Free.