A/B Testing: FAQ

Here you will find the answers to the following questions concerning A/B Testing in Woodpecker, for example, why should you test your campaign, how to quickly check if the campaign was A/B tested, how to turn off the sending email version.

Why should I use A/B testing? How am I going to benefit from this test?

A/B testing shows you which variation of your email proves to be the most fruitful and helpful. While experimenting with the subject line or email copy, you learn which of your practices brings better results. In turn, you discover how to tweak your campaigns to turn your cold contacts into business partners. Also, your chances to avoid being caught by anti-spam filters are higher when using multiple versions of your email. The more variations of email and the more randomized sending, the better for your email deliverability.

I'm on a Start-up plan. Can I use the A/B testing?

A/B Testing is available on Team Pro or Agency plan. Learn how to upgrade your plan »

What can I test?

Anything that you have in the content of your email. You can experiment with multiple variations of the subject line and the email body. Delivery time that you set up for each step (each email in the campaign sequence) and your signature will stay the same. 

Is there a way to quickly check if I used A/B testing in my campaign?

Yes, look for an ‘A/B’ tag on the list of your campaigns. Split test campaigns are marked with the ‘A/B’ tag.

How many versions of my email can I test?

You can add and test up to 5 versions: A, B, C, D, E. If you’re not happy with the version you created, you can delete it and then add a new one.

How can I make the subject line feel more personal? 

To personalize your subject line you can use one of the snippets available in the email editor, for example, {{FIRST NAME}} or {{COMPANY}}. There’s plenty of options. 

Before you hit the ‘SEND’ button, though, make sure you’re not missing any important information. Preview your campaign and send a test campaign to yourself and to see how your emails will look like in your prospect’s inbox. Read more about snippets »

Can I also A/B test follow-up emails? 

Yes, you can carry out the A/B test for all steps in the campaign. The process is fairly the same as when you split test the opening email. Add a blank new version by clicking the green  + symbol (‘Add version’) or copy the existing version. One version of a follow-up will be randomly assigned to a prospect in each step. 

How does it work? Let’s say you have 1 opening email and 2 follow-ups in the campaign sequence. That gives 3 steps: 

  • Email 1 (opening email) without A/B testing.
  • Email 2 (follow-up) with A/B testing.
  • Email 3 (follow-up) also with A/B testing. 

A new version will be drawn for a prospect right before sending it. That means that your prospect can receive, for instance, version B of Email 2 and version A of Email 3.

Can I perform A/B testing on a campaign with conditions?

Yes, there are no restrictions. Please mind that although we don’t block performing A/B tests in If-campaigns, the statistics may not be accurate due to a small number of prospects in each Path.

Is there a minimum number of prospects needed for the A/B test?

No, there isn’t. Woodpecker won’t block the sending even if you have only 2 or 3 prospects in the campaign. Nevertheless, we recommend sending one email version to at least 100 prospects. So if you’re planning to test 4 versions of email, it’s best to import at least 400 prospects.

How are the prospects assigned to one version?

Prospects are selected in random order from the pool of email addresses added to the campaign. First, the prospect will be drawn for the version A of Email 1, second, for version B, thirdly, for version C, and so on. Next, in the same process, we will randomly select prospects for the next step in a campaign.

How many prospects are assigned to one version? Do you assign prospects evenly?

Prospects are selected randomly yet distributed evenly. That means that each version should be sent to a more or less the same number of prospects. 

What about the statistics? Will they show that I used A/B testing in the campaign?

Yes, open your campaign and have a look at your stats dashboard. There you’ll see all versions you have tested. Your best results will be marked with a little badge – a special winner icon. For example, your open rate may be the best in version A (check the column ‘OPENED’) while the highest response rate may bring version C (‘RESPONDED). 

‘Best result’ badge for Interest level is available only for the prospects marked as 'Interested'.    

If you’re not happy with the performance of some of the email versions, you can turn them off so they won’t be sent to prospects anymore.

Can I turn off the sending email version?

Yes, you can turn off an email version with worse results so it won’t be sent to prospects anymore. Please note that at least one version of the sending email has to be active. 

Open your stats dashboard and click the toggle switch to turn them off/on. 

  • Versions that you turned off are marked in grey color.
  • Active versions are marked in green color. 
  • If you decide to turn off sending versions with customized emails, those emails will still be sent.

Can I delete a version?

Deleting versions is possible as long as no emails have been scheduled to be sent. What does it mean and how does it work? You can see scheduled emails as QUEUED emails in the statistics. 

Please note that the option to delete a version will be disabled if at least one version has been sent from a particular step. You’ll see a message “You can’t delete any version as one of them has already been sent”.

Can I customize an email while doing A/B testing?

When previewing your campaign you’ll see the option to customize Email 1. If you edit one of the versions, that version will be assigned to a prospect. Other versions of Email 1 won’t be shown in the preview. See the example below.

Can I edit my email versions after the sending has started?

Yes, for example, when you notice that the performance and conversion rate of one of the sending email versions is low.

Can I add a new version when my campaign is running?

Yes, you can add another version at any time.

What’s the difference between ‘Send test email’ and ‘send test campaign’?

‘Send test email’ will send all versions of that email. We’ll inform you about it. If you have 5 versions of the opening email, we will send all of them.  

‘Send test campaign’, however, will result in the randomized sending of several versions to your email address. 

Why did we decide to slightly change this process in A/B testing? Imagine having 5 emails in your sequence, each testing 5 different subject lines. Your inbox would get instantly cluttered with 25 test emails.  

It seems that I cannot add any version. Why?

A/B testing is possible only on new If-campaigns. If you’re a long time user, you may be using the old versions of our campaigns. image. Please switch to If-campaigns to use A/B testing. 

Go to Settings → If-campaigns and toggle the switch to activate the new feature. Learn more about If-campaigns »

What happens to the versions when I copy the whole campaign?

Making a copy Woodpecker will also replicate all the added versions except the ones which you customized in preview. Learn more about customizing email in the preview »

How do conditions work when I carry out the A/B is some versions?

In case you added an open-based condition – all of the version needs to have open tracking allowed. We’ll show you a short reminder in case you forget about switching the open tracking on: ‘Open tracking needs to be on because of the condition you’ve set for the next follow-up”.

The same goes for click-based conditions. We’ll notify you that you need to ‘Add a tracking link to all versions of the previous email’. Otherwise, sending will be impossible.

Did this answer your question? Thanks for the feedback There was a problem submitting your feedback. Please try again later.

Still need help? Contact Us Contact Us