A message must never go out twice: how we built sending in our outreach tool

Nothing exotic has to go wrong for an email to go out twice. A worker restarts halfway through a batch. A job queue delivers the same job again. Someone runs the import a second time because the first run looked stuck. A call to the mail server times out and the code retries. All of that is ordinary, and a sender that only works when every step runs exactly once will fail on an ordinary day.

So I don't treat a duplicate as a bug to catch in testing. I assume every step will run twice at some point, and I design so that the second run does nothing. Engineers call that property idempotency: doing a thing twice has the same effect as doing it once.

Where a rule lives matters as much as what it says. Application code has bugs, and two copies of it can run at the same moment. A rule in the database holds whichever code path, worker or person at a console pushes against it. All three rules below live there.

In our tool an email is a row in a table. The row says which lead it's for and which step of the sequence it is: the first message, the first follow-up and so on. The database allows only one row per lead and sequence step. That's a uniqueness rule, and a second insert for the same pair is refused.

That one rule covers a lot of ground. An import that runs again can't add step 2 for the same lead a second time. A retry of the code that queues emails can't either, and neither can a person fixing something by hand.

If you don't write code, think of a guest list where each name may appear once. The rule sits on the list. It doesn't depend on the person at the door remembering who they've already let in.

It also gives everyone one place to look. When I ask "did this lead get step 2?", the answer is the status of one row.

The risky moment in any sender is the move from "waiting" to "being sent". The naive version takes two steps: read the rows that are queued, then mark them as taken. Two workers that start together can both finish the read before either does the mark. Each believes the row is its own, and both send. I call that the check-then-send gap.

Our worker closes the gap by doing both steps in one statement. It switches a batch of rows from queued to sending, and the rows are locked as they're picked. In Postgres the piece that does this is SELECT ... FOR UPDATE SKIP LOCKED. FOR UPDATE locks every row the statement selects. SKIP LOCKED tells it to pass over any row another transaction already holds, where the default would be to wait for that row.

Run two workers at once. The first locks ten rows and flips them to sending. The second, a few milliseconds behind, skips those ten and takes the next ten. Once the first worker commits, its rows are no longer queued, so nobody picks them up later either. Ask "what happens when this runs twice?" of the claim step and the answer is that the second run gets different rows, or none.

The Postgres documentation is frank about the trade. Skipping locked rows gives an inconsistent view of the data, so it isn't for general-purpose work. The use it does name is several consumers reading a queue-like table, and that's what this table is.

The snippet below shows the shape of such a claim. It's illustrative: the table and column names are made up for this post.

-- Illustrative only. Claim up to 10 queued emails in one statement.
UPDATE emails
SET status = 'sending'
WHERE id IN (
  SELECT id
  FROM emails
  WHERE status = 'queued'
  ORDER BY id
  LIMIT 10
  FOR UPDATE SKIP LOCKED
)
RETURNING id;

Claiming settles who owns a row. The send log settles whether the email went out. The log has its own uniqueness rule: exactly one entry per email row. The database refuses a second entry for the same row, the same way it refuses a second row for the same lead and step.

Every send also carries an idempotency key. That's a value that identifies one specific send, so that when the same operation arrives twice, the receiving side can tell it's the same one and not a new one. The key belongs to the row and doesn't change between attempts.

Here's the case this is for. The worker hands a message to the mail server and the call times out. Did it go? A sender with no log has to guess, and both guesses are bad: retry and risk a duplicate, or skip and risk a gap. With a one-entry log and a fixed key, a retry is the same send arriving again, and there's a record to check first.

One limit, stated plainly. If a worker dies in the instant after the mail server accepts a message and before that fact is written down, the database alone can't know which side of the line it died on. So our tool records the send only after the mail server has accepted it, and treats a row that has sat in sending for more than 30 minutes as unconfirmed: it is marked failed with the note "check Sent Items before doing anything", and it is never retried automatically. A person looks first. A send that fails with a clear, retryable error is different: it goes back to the queue 30 minutes later, at most three attempts in total.

Some emails must not go out even once. The lead unsubscribed, say, or the address is on a do-not-contact list. Those addresses are suppressed.

We check suppression at three moments: when a lead is added, when an email is queued, and right before the email is handed to the mail server. It takes three because the answer changes over time. A lead can be fine on Monday when the import runs and unsubscribe on Wednesday, after the follow-up is already queued. Only a check at the last moment catches that.

All three checks call the same database function, and I'd give up the count before I gave up that. If the import, the queue and the sender each carry their own copy of the logic, the copies drift: someone changes the rule in one place and forgets the other two. With one function, a change lands everywhere at once.

A made-up example. A lead at a regional bakery chain is imported, passes the check and gets the first email. She clicks unsubscribe. Step 2 is already sitting in the queue for Thursday. On Thursday a worker claims the row, calls the function once more before the handoff, gets a no, and the email isn't sent.

None of this helps if the mail lands in spam. Google publishes sender guidelines for Gmail, and since 1 February 2024 they apply to everyone who sends to Gmail accounts.

For all senders the list includes SPF or DKIM authentication on the sending domain, valid forward and reverse DNS records, a TLS connection, and a spam rate below 0.3% as reported in Google's Postmaster Tools. SPF says which servers may send for your domain. DKIM signs each message so the receiver can verify where it came from.

Send more than 5,000 messages a day to Gmail accounts and Google's bulk sender rules apply too. You need SPF, DKIM and DMARC together (the DMARC policy may be set to none), and the domain in your From address has to align with the SPF or DKIM domain. Marketing and subscribed messages must support one-click unsubscribe and show a clearly visible unsubscribe link in the body. One-click means two headers on the message: List-Unsubscribe, and List-Unsubscribe-Post: List-Unsubscribe=One-Click. On spam rate, Google's advice is to stay below 0.10% and never reach 0.30%.

The bulk rules only bind above 5,000 a day. I build to them from the first email anyway: all three authentication records, both unsubscribe headers and a visible link. An unsubscribe lands in the suppression data that the function above reads.

Pace is the last piece. A new mailbox has no reputation yet, so we start each one at no more than 10 emails a day in its first week. The spam rate I aim for is the stricter one, under 0.1%. A duplicate email is an easy way to earn a spam click, so the work in the earlier sections counts here too.

You don't need to read SQL to check a sender. Ask the person who built it four questions and listen for where the answers point.

• What happens when this runs twice? A good answer names something in the database that refuses the second attempt: a uniqueness rule, a lock, a log with one entry per email. A weak answer is about timing, like "the workers are staggered" or "we wait a few seconds".
• When is suppression checked, and by what? You want to hear "right before sending" somewhere in the answer, and you want one shared check instead of a copy per feature.
• Is the sending domain authenticated, and do emails carry one-click unsubscribe? SPF, DKIM and DMARC are DNS records and the unsubscribe headers are in every message, so someone can show them to you.
• Where do we see the spam rate, and how fast do new mailboxes ramp up? For Gmail the place is Postmaster Tools. The numbers to hold against it are 0.3% as the limit and 0.1% as the target.

If you have a codebase that sends email, a coding agent such as Claude Code can answer those questions from the code. Give it read-only access and paste the prompt below. It asks for file paths so a developer can verify each answer.

Read the report as a work list. If reading and claiming are two steps, they need to become one statement. If the send log has no uniqueness rule, add one. If suppression is checked in several places by different code, move it into one function. I do this kind of work under product engineering. If you'd like a second pair of eyes on the report, start at for founders or get in touch.

Goal: review this repository's email send path for duplicate sends and suppression gaps. Report only. Change nothing.

Inputs: read-only access to the repository, including the database schema and migrations, the code that queues emails, the worker that claims and sends them, the send log, the suppression logic and the call to the mail provider.

Constraints: do not edit files, run migrations, send mail or call external services. Use only what you find in the repository. If something is not in the repository, write "not found" and do not guess.

Definition of done: a short report that answers these questions with file paths and line numbers:
1. Where is an email stored, and what stops a second row for the same recipient and sequence step?
2. Quote the statement that claims emails for sending. Are reading and claiming one statement or two?
3. What in the database refuses a second send-log entry for the same email?
4. Is there an idempotency key? Where is it created and where is it passed on?
5. At which points is suppression checked, and do all of them call the same function?
6. For the claim step, the handoff to the mail provider and the logging step: what happens when it runs twice?
End with a list of red flags, most serious first.