Experiment Type: "Design"
|Time: 10 minutes|
|Suggested Skills: Statistics|
|Objective: This article outlines how to set up an Experiment campaign with variations of the design going to your contacts.|
Go to Experiments > New Experiment
Name Your Experiment
Select "Design" as your experiment type
Create your campaign
Set your variations
- The Description field is an internal description of this version.
- Use the 'Add Variation' button to add a new variation.
- Use the quick test to receive all variations in your inbox.
- Actions include:
|Edit this variation design|
|Edit campaign details (applies to whole Experiment)|
|Preview and test this variation only|
|Edit trackable links in this variation only|
|Edit social sharing settings in this variation only|
|Remove this variation|
Select your recipients and sample size
The experiment will default to list all contacts within your experiment. Should you want to send to specific lists or segments, select them in this step. What is Failsafe Sending?
Select either the percentage or number of contacts you'd like to be sent variations. The number of variations you have will be split up evenly among your sample contacts. (# of sample contacts / # of variations) For example, if your sample size is 50 contacts, and you have 4 variations, about 12 contacts will be sent Version A, another 12 will be sent Version B and so forth. Refer to How To: Create an Experiment to ensure your experiment is statistically significant. The remainder segment will receive the "winner", based on the metric you set within the timeframe in step 3.
Set your experiment length and metric
You may run the experiment for 1 to 23 hours. Your experiment length may be a neutral item, however it may also influence the results of your experiment significantly. For example, think about the time of day you are starting your experiment. Does it correlate to a time of day in which your contacts are more likely to open/click your emails? Or is it a time of day in which they are less likely? This will play a role in determining the ideal experiment length. The more popular the hour, the shorter the experiment can be and alternatively, the less popular, the longer the experiment should be. mail2 recommends 23 hours to give your experiment a full day cycle to collect as much sample data as possible. For Design-based experiments, we recommend setting clicks as your metric, either unique, total or specific rather than opens.
By sending a quick test, test recipients will receive a copy of each variation to review the email before the experiment begins.
Save and Close
Schedule the Experiment
Set when your experiment will begin, either immediately or at a later date and time.
Set when the 'winner' will send, either immediately after the experiment is over (based on the amount of time you set in Step 7) or at a certain date and time (after the experiment) or be notified and manually deploy the winner.