Experiment Type: "Subject"

Knowledge Level: 
Time: 10 minutes
Suggested Skills: Statistics
Objective: This article outlines how to set up an Experiment campaign with variations of the subject line going to your contacts. 


Go to Experiments > New Experiment


Name Your Experiment


Select 'Subject' as your experiment type


Create your campaign


Set your variations

  • The Description field is an internal description of this version.
  • The Subject is what your patrons will see when they receive the email. You'll use this field to set the variances on your experiment. 
  • Use the 'Add Variation' button to add a new variation. 
  • Use the quick test to receive all variations in your inbox. 
  • Actions include:
Edit this variation design
Edit campaign details (applies to whole Experiment)
Preview and test this variation only
Edit trackable links in this variation only
Edit social sharing settings in this variation only
Remove this variation

Select your recipients and sample size

The experiment will default to list all contacts within your experiment. Should you want to send to specific lists or segments, select them in this step. What is Failsafe Sending?

Select either the percentage or number of contacts you'd like to be sent variations. The number of variations you have will be split up evenly among your sample contacts. (# of sample contacts / # of variations) For example, if your sample size is 50 contacts, and you have 4 variations, about 12 contacts will be sent Version A, another 12 will be sent Version B and so forth. Refer to Creating an Experiment to ensure your experiment is statistically significant. The remainder segment will receive the "winner", based on the metric you set within the timeframe in step 3.


Set your experiment length and metric

You may run the experiment for 1 to 23 hours. Your experiment length may be a neutral item, however it may also influence the results of your experiment significantly. For example, think about the time of day you are starting your experiment. Does it correlate to a time of day in which your contacts are more likely to open/click your emails? Or is it a time of day in which they are less likely? This will play a role in determining the ideal experiment length. The more popular the hour, the shorter the experiment can be and alternatively, the less popular, the longer the experiment should be. mail2 recommends 23 hours to give your experiment a full day cycle to collect as much sample data as possible. 



By sending a quick test, test recipients will receive a copy of each variation to review the email before the experiment begins. 


Save and Close


Schedule the Experiment

Set when your experiment will begin, either immediately or at a later date and time.

Set when the 'winner' will send, either immediately after the experiment is over (based on the amount of time you set in Step 7) or at a certain date and time (after the experiment) or be notified and manually deploy the winner. 

Still need help? Contact Us Contact Us