Guide

How to A/B Test Your Follow to DM Messages

Most people set up one Follow to DM message and never change it. That means they are stuck with whatever reply rate they got on day one. A/B testing fixes that. Here is how to split test your welcome messages using ManyChat and DM Tracker.

Quick answer: Use ManyChat’s Randomizer to split new followers between different welcome messages. Connect DM Tracker to track which variant gets more replies. Run each test for 100+ followers per variant (usually 2-4 weeks). Keep the winner, test a new challenger, repeat. This is how you systematically increase your Follow to DM reply rate over time.

Why Most People Never A/B Test (and Why They Should)

Here is the typical Follow to DM setup:

Someone sets up ManyChat Follow to DM
They write a welcome message
They publish it
They never change it again

Six months later, they have no idea if their message is performing well or terribly. They do not know their reply rate. They have never tested an alternative. They are stuck with the same results from day one.

The problem is not laziness. The problem is that ManyChat does not track reply rates.

ManyChat shows you how many messages were sent. But it does not show how many people replied. So there is no feedback loop. No signal telling you “this message is working” or “this message needs improvement.”

A/B testing solves this, but only if you have a way to measure the results. That is where DM Tracker comes in.

The A/B Testing Framework

Here is the complete system:

ManyChat Randomizer splits new followers between message variants
DM Tracker tracks reply rates for each variant
You compare the data and keep the winner
Repeat with a new challenger variant

This creates a continuous improvement loop. Every test makes your Follow to DM message a little better. Over 3-6 months, your reply rate can improve by 15-25 percentage points.

Step 1: Set Up the ManyChat Randomizer

In your existing Follow to DM flow:

Open your Follow to DM automation in ManyChat
Click the connection between the trigger and your first message
Add a Randomizer block (found in the flow builder under “Logic”)
Set the split ratio:
- Two variants: 50% / 50%
- Three variants: 33% / 33% / 34%
Connect each Randomizer output to a different message block

What the Flow Looks Like

[Follow to DM Trigger]
       |
   [Randomizer]
    /        \
[Message A] [Message B]
  (50%)       (50%)

Each new follower gets randomly assigned to one variant. Over time, the split will be roughly even.

Step 2: Write Your Test Variants

The key to good A/B testing is changing one variable at a time. If you change the format, tone, and length all at once, you will not know which change made the difference.

Test 1: Message Format (Start Here)

This is the highest-impact test. Three main formats to compare:

Variant A: Question Opener

Hey! Thanks for the follow. Quick question: are you looking to grow your business with Instagram, or just getting started?

Variant B: Value Offer

Hey, welcome! I put together a free guide on Instagram DM selling that I think you will find useful. Want me to send it over?

Variant C: Curiosity Hook

Hey! Just noticed you followed. Curious, what made you hit follow? Always love hearing what people are working on.

Run all three (or pick two) and see which format resonates with your audience. The winner becomes your baseline for future tests.

Test 2: Specificity

Once you know the winning format, test how specific your message should be.

Variant A: Broad

Hey! Thanks for the follow. What are you working on right now?

Variant B: Specific

Hey! Thanks for the follow. Are you looking to book more discovery calls from Instagram, or are you focused on growing your audience first?

Specific messages typically get higher reply rates because they show the follower you understand their situation.

Test 3: Tone

Variant A: Professional

Welcome! I help coaches scale their business through Instagram DMs. What brings you to our page?

Variant B: Casual

Hey! Thanks for the follow. What is your biggest challenge with Instagram right now?

Test 4: Emoji Usage

Variant A: No emoji

Hey! Thanks for the follow. Quick question: are you looking to scale your coaching business?

Variant B: With emoji

Hey! Thanks for the follow. Quick question: are you looking to scale your coaching business?

Yes, something this small can affect reply rates. Test it.

Test 5: Message Length

Variant A: Ultra-short (1 sentence)

Hey! What are you working on right now?

Variant B: Standard (2-3 sentences)

Hey! Thanks for the follow. I noticed you are into [topic]. Quick question: are you looking for help with [specific outcome], or are you still figuring things out?

Test 6: Buttons vs No Buttons

Variant A: Text only

Hey! Thanks for the follow. Are you looking to grow your business or just exploring?

Variant B: With quick-reply buttons

Hey! Thanks for the follow. What brings you here? [Button: “Looking to grow my business”] [Button: “Just exploring”]

Buttons reduce friction (tapping is easier than typing), but some audiences prefer typing their own reply.

Step 3: Tag Variants in DM Tracker

This is the step that makes the whole system work.

When a new follower enters your Follow to DM flow, the ManyChat Randomizer assigns them to a variant. In DM Tracker, you need to identify which variant each follower received.

Method 1: Manual Tagging

Open each new conversation in DM Tracker and add a tag based on the message they received:

Tag: “Welcome A” (for the question opener)
Tag: “Welcome B” (for the value offer)

This works for smaller volumes but gets tedious above 50+ conversations per week.

Method 2: ManyChat Tags + DM Tracker Tags

In ManyChat, add a tag action in each Randomizer branch. For example:

Branch A: Add ManyChat tag “Variant A”
Branch B: Add ManyChat tag “Variant B”

These ManyChat tags flow through to DM Tracker, making it easy to filter.

Method 3: Unique Keywords

Add a unique, subtle identifier to each variant. For example:

Variant A: “Hey! Quick question: are you looking to grow your business or just getting started?”

Variant B: “Hey! Curious: are you looking to grow your business or just getting started?”

The only difference is “Quick question” vs “Curious.” When you see the conversation in DM Tracker, you know which variant they received.

Step 4: Let It Run

Do not check results daily. Early data is noisy. A handful of replies can swing your percentages wildly.

Here is the timeline:

Days 1-3: Too early. Ignore the data.
Week 1: Getting some data, but still not enough.
Week 2-3: Starting to see patterns. Do not make changes yet.
Week 4+ (or when you hit 100 followers per variant): Time to analyze.

Minimum Sample Size

For each variant, you need at least 100 followers before the data becomes reliable. Here is why:

With 20 followers per variant, one extra reply changes your reply rate by 5%.
With 50 followers per variant, one extra reply changes your reply rate by 2%.
With 100 followers per variant, one extra reply changes your reply rate by 1%.

The larger your sample, the more confident you can be in the results.

How Long That Takes

New Followers/Week	2-Variant Test	3-Variant Test
50/week	4 weeks	6 weeks
100/week	2 weeks	3 weeks
200/week	1 week	1.5 weeks
500/week	3 days	4 days

Step 5: Analyze Results in DM Tracker

Once you have enough data, go to DM Tracker’s statistics dashboard and filter by your variant tags.

What to Compare

Primary metric: Reply rate

This is the most important number. Which variant gets a higher percentage of followers to reply?

Example results:

Variant A (Question opener): 38% reply rate (38 replies out of 100 followers)
Variant B (Value offer): 45% reply rate (45 replies out of 100 followers)
Variant C (Curiosity hook): 33% reply rate (33 replies out of 100 followers)

Winner: Variant B. The value offer resonated best with this audience.

How to Know If the Difference Is Real

A general rule:

5%+ difference with 100+ followers per variant = meaningful, act on it
3-5% difference = possibly real, run for another week to be sure
Under 3% difference = likely noise, either run longer or call it a tie

Secondary Metrics to Check

Beyond reply rate, look at:

Conversation depth: Do replies from one variant lead to longer conversations?
Follow-up success: Which variant’s leads are more likely to re-engage after going quiet?
Quality of replies: Are people writing thoughtful responses, or just one-word answers?

These secondary signals help you understand not just who replies, but who becomes a real conversation (and eventually a customer).

Step 6: Implement and Iterate

Once you have a winner:

Pause the losing variants in ManyChat
Run 100% traffic to the winning message
Wait 1-2 weeks to confirm the winning reply rate holds
Test a new challenger against the current winner

This creates a continuous improvement cycle:

Month 1: Test format (question vs value offer vs curiosity hook). Winner: Value offer at 45% reply rate.
Month 2: Test specificity (broad value offer vs specific value offer). Winner: Specific at 49% reply rate.
Month 3: Test tone (professional vs casual). Winner: Casual at 52% reply rate.
Month 4: Test buttons vs no buttons. Winner: Buttons at 55% reply rate.

Over four months, your reply rate went from a baseline of 38% to 55%. That is a 45% improvement in conversations started, all from testing.

What to Test: Priority Order

If you are just starting A/B testing, follow this order:

Priority 1: Message Format

Question opener vs value offer vs curiosity hook. This is the biggest lever.

Priority 2: Specificity

Broad message vs industry/niche-specific message. Alignment with your audience matters.

Priority 3: Call-to-Action Type

Buttons vs text reply vs link. How people respond affects whether they respond.

Priority 4: Tone and Voice

Professional vs casual vs enthusiastic. Match your brand personality.

Priority 5: Timing

Instant vs 1-minute delay vs 2-minute delay. Test this separately from message content.

Priority 6: Small Details

Emoji vs no emoji, first name personalization, message length. These are micro-optimizations once you have the big things dialed in.

Common A/B Testing Mistakes

Mistake 1: Ending Tests Too Early

You see Variant A at 50% reply rate after 15 followers and declare it the winner. Then over the next 100 followers, it drops to 30%. Small samples are unreliable. Wait for 100+ per variant.

Mistake 2: Changing Multiple Variables at Once

You test a short, casual, emoji-filled message against a long, professional, text-only message. It wins. But what actually caused the improvement? You have no idea. Change one thing at a time.

Mistake 3: Never Testing

The biggest mistake is skipping A/B testing entirely. Your first message is almost never your best message. Even a 5% improvement in reply rate compounds over thousands of followers.

Mistake 4: Testing Without Tracking

Running ManyChat’s Randomizer without DM Tracker means you are splitting traffic but not measuring results. The Randomizer creates the experiment. DM Tracker provides the answer. You need both.

Mistake 5: Optimizing for Replies, Not Conversations

A clickbait-style message might get high reply rates but lead to low-quality conversations. Look beyond the reply rate. Check if the replies are leading to real conversations and follow-ups.

The Bottom Line

A/B testing Follow to DM messages is the single fastest way to improve your DM conversion rate. The framework is simple:

Split traffic with ManyChat’s Randomizer
Track reply rates with DM Tracker
Keep the winner, test a new challenger
Repeat every 2-4 weeks

Most ManyChat users never do this because they have no way to track reply rates. DM Tracker changes that. $39/user/month, 14-day free trial, 5-minute setup.

Start with one test. See which message wins. Then keep testing. Your reply rate will thank you.

Related guides:

Frequently Asked Questions

Partially. ManyChat has a Randomizer feature that splits new followers between different message variants. But ManyChat cannot tell you which variant gets more replies. You need DM Tracker to track reply rates per variant and determine the winner.

You need at least 100 followers per variant for meaningful results. For a two-variant test, that means 200 new followers total. For a three-variant test, 300. Below these numbers, random variation can make the results unreliable.

Run each test until you have at least 100 followers per variant. For most accounts, this takes 2-4 weeks. Do not end a test early based on small sample sizes, even if one variant looks like it is winning.

Start by testing the message format: question opener vs value offer vs curiosity hook. This is the highest-impact variable. Once you find the winning format, test specific wording, emoji usage, button options, and timing.

DM Tracker tracks reply rates for all outreach, including Follow to DM conversations. To compare A/B test variants, tag each conversation with its variant name (e.g., 'Welcome A', 'Welcome B') and filter by tag to compare reply rates.

A difference of 5% or more with 100+ followers per variant is generally meaningful. For example, if Variant A gets 32% reply rate and Variant B gets 41% reply rate over 100+ followers each, Variant B is the clear winner. Differences under 3% are likely noise.

Yes. ManyChat's Randomizer supports multiple branches. You can test 2, 3, or even 4 variants simultaneously. However, more variants means you need more followers to reach meaningful sample sizes. For most accounts, 2-3 variants is the sweet spot.

Yes, but test it separately from message content. Change one variable at a time. First, find your best message. Then test timing (instant vs 1-minute vs 2-minute delay) to optimize delivery.