Batch Processing Large Email Lists

When your email list exceeds your hourly sending limit, you need to break it into batches and submit them progressively — either spread across hours or days. This article explains how to structure your code to handle large list processing efficiently, safely, and with proper error handling and retry logic.

Why You Need Batch Processing

Recommended Batch Architecture

1. Chunk your list — divide recipients into batches matching your hourly limit (e.g. 150 per batch for the 30k plan).
2. Schedule each batch — use the send_at parameter to schedule each chunk at the appropriate time (current hour, next hour, etc.).
3. Track submission state — store the batch ID and status for each chunk so you can resume if the process is interrupted.
4. Handle errors per batch — catch failures at the batch level, not the individual recipient level, for efficiency.
5. Collect and process results — use webhooks to update your records as each batch delivers.

Complete Batch Processing Implementation — Python

import os
import time
import requests
from datetime import datetime, timedelta, timezone

API_KEY    = os.environ['MIGOSMTP_API_KEY']
API_URL    = 'https://api.migosmtp.com/v1/email/send/bulk'
HOURLY_CAP = 150   # 30k plan hourly limit
CHUNK_SIZE = 150   # Batch size per submission

def send_campaign(recipients, subject, html_body, text_body):
    """
    Send a campaign to a large recipient list in scheduled hourly batches.
    recipients: list of dicts with 'to' and optional 'variables'
    """
    chunks      = [recipients[i:i+CHUNK_SIZE] for i in range(0, len(recipients), CHUNK_SIZE)]
    total       = len(recipients)
    results     = []
    send_time   = datetime.now(timezone.utc)

    print(f"Sending {total} emails in {len(chunks)} batches of {CHUNK_SIZE}")

    for index, chunk in enumerate(chunks):
        # Schedule each batch one hour apart
        scheduled_for = send_time + timedelta(hours=index)

        payload = {
            'from':       'newsletter@yourcompany.com',
            'subject':    subject,
            'html':        html_body,
            'text':        text_body,
            'recipients': chunk,
            'send_at':    scheduled_for.strftime('%Y-%m-%dT%H:%M:%SZ'),
            'tags':       [f'campaign:batch-{index+1}'],
        }

        try:
            response = requests.post(
                API_URL,
                headers={
                    'Authorization': f'Bearer {API_KEY}',
                    'Content-Type':  'application/json',
                },
                json=payload,
                timeout=30
            )
            response.raise_for_status()
            data = response.json()

            results.append({
                'batch':        index + 1,
                'count':        len(chunk),
                'batch_id':     data.get('batch_id'),
                'scheduled_for': scheduled_for.isoformat(),
                'status':       'scheduled'
            })
            print(f"Batch {index+1}/{len(chunks)}: {len(chunk)} emails scheduled for {scheduled_for.strftime('%H:%M UTC')}")

        except requests.exceptions.HTTPError as e:
            error_body = e.response.json() if e.response else {}
            results.append({
                'batch':  index + 1,
                'count':  len(chunk),
                'status': 'failed',
                'error':  error_body.get('message', str(e))
            })
            print(f"Batch {index+1} failed: {error_body.get('message', str(e))}")

        except requests.exceptions.Timeout:
            results.append({'batch': index + 1, 'status': 'timeout'})
            print(f"Batch {index+1} timed out — will need manual retry")

        # Small delay between API submissions to avoid API rate limit
        time.sleep(0.5)

    successful = sum(1 for r in results if r['status'] == 'scheduled')
    failed     = sum(1 for r in results if r['status'] != 'scheduled')
    print(f"
Summary: {successful} batches scheduled, {failed} failed")
    return results

Complete Batch Processing — Node.js

const axios = require('axios');

const API_KEY    = process.env.MIGOSMTP_API_KEY;
const CHUNK_SIZE = 150;
const DELAY_MS   = 500; // 500ms between batch submissions

function chunkArray(arr, size) {
  return Array.from({ length: Math.ceil(arr.length / size) },
    (_, i) => arr.slice(i * size, i * size + size));
}

function addHours(date, hours) {
  return new Date(date.getTime() + hours * 3600000).toISOString().replace('.000Z', 'Z');
}

async function sleep(ms) {
  return new Promise(resolve => setTimeout(resolve, ms));
}

async function sendCampaignInBatches(recipients, subject, html, text) {
  const chunks    = chunkArray(recipients, CHUNK_SIZE);
  const baseTime  = new Date();
  const results   = [];

  console.log(`Sending ${recipients.length} emails in ${chunks.length} batches`);

  for (let i = 0; i < chunks.length; i++) {
    const chunk       = chunks[i];
    const scheduledAt = addHours(baseTime, i);

    try {
      const { data } = await axios.post(
        'https://api.migosmtp.com/v1/email/send/bulk',
        {
          from:       'newsletter@yourcompany.com',
          subject,
          html,
          text,
          recipients: chunk,
          send_at:    scheduledAt,
          tags:       [`batch:${i + 1}`],
        },
        {
          headers: {
            'Authorization': `Bearer ${API_KEY}`,
            'Content-Type':  'application/json',
          },
          timeout: 30000,
        }
      );

      results.push({ batch: i + 1, batchId: data.batch_id, status: 'ok', scheduledAt });
      console.log(`Batch ${i + 1}/${chunks.length}: ${chunk.length} emails → ${scheduledAt}`);

    } catch (err) {
      const msg = err.response?.data?.message || err.message;
      results.push({ batch: i + 1, status: 'error', error: msg });
      console.error(`Batch ${i + 1} error: ${msg}`);
    }

    if (i < chunks.length - 1) await sleep(DELAY_MS);
  }

  return results;
}

Tracking and Resuming Failed Batches

Store batch submission results in your database so you can identify and retry failed batches:

// Pseudocode — save batch state
for each batch in results:
    db.save({
        campaign_id:   'summer-sale-2026',
        batch_number:  batch.number,
        batch_id:      batch.api_batch_id,
        recipient_ids: batch.recipient_ids,
        status:        batch.status,  // 'scheduled' | 'failed' | 'delivered'
        scheduled_at:  batch.scheduled_for,
        created_at:    now()
    })

// To resume: query for status = 'failed' and retry those batches only

Error Handling Strategy for Batch Sends

Next Steps

• Best practices for high-volume sending
• Email queue management
• Understanding rate limits & throttling