Capacity planning for sending and receiving email
Capacity planning for sending and receiving email
Sending and receiving email at low to moderate volume and scale is often straightforward. However, at high volume and scale there are additional aspects to consider, which require a bit of planning to determine the best configuration to meet your specific needs. This article will outline some of the considerations and configuration options that together will help you achieve these goals.
Before you begin
Please familiarize yourself with the About Email article on Pega Community.
Inbound Email
When planning inbound email requirements, the key components to understand are:
- How many email inboxes will you need to receive inbound email from?
- What is the peak, average and daily inbound volume that is seen on average in each inbox?
- How quickly do emails needs to be processed and available to an agent?
- How long does it take, on average, to process and create the corresponding case/cases for the email?
- What is the sleep time between runs, and the maximum number of records to process for each run?
The following table of examples takes the factors listed above and shows the average and peak listener thread concurrency needed to process emails within 1 hour of being received by the mail provider.
- Concurrent threads per hour uses 3,600 seconds in an hour, divided by average time per message.
- The 3 main factors that will drive your throughput are the time it takes to process an individual message, the max messages per iteration, and the sleep period between cycles. Try playing with the numbers and see if your processing is bound by the maximum messages or by time by adjusting the columns.
- When planning your capacity, always include additional buffer capacity. The examples below assume 20%. Numbers should be rounded up.
| Email Account Name | Peak Inbound Volume
per hour |
Average Daily Volume | Average processing
time in seconds for an individual message |
Maximum Number
of messages to process before latency period |
Latency/Sleep
period in seconds |
Max Processing
Time in seconds |
Seconds Available
Per Hour to Process |
Processing Capacity
Per Hour |
Concurrent Listener
threads required |
With 20%
Capacity Buffer |
| sales@pegaSample.com | 850 | 6,120 | 3.1 | 500 | 100 | 500 | 3000 | 968 | 1 | 1 |
| offers@pegaSample.com | 12,000 | 86,400 | 3.1 | 500 | 100 | 500 | 3000 | 968 | 12 | 15 |
| marketing@pegaSample.com | 2,800 | 20,160 | 3.1 | 500 | 100 | 500 | 3000 | 968 | 3 | 3 |
| members@pegaSample.com | 300 | 2,160 | 5 | 500 | 100 | 500 | 3000 | 600 | 1 | 1 |
| support@pegaSample.com | 6,500 | 46,800 | 3.1 | 500 | 100 | 500 | 3000 | 968 | 7 | 8 |
The screenshot below shows the email listener configuration properties that drive throughput.
Once you have determined the maximum number of threads required, with a buffer, there are a few further configuration options to keep in mind:
- When setting up your listeners for a high volume inbound situation, ensure that you segment the background processing of inbound email from interactive user sessions to avoid performance contention. One option for email listeners is the ability to control which nodes or what class of node the listener threads will run on. In the screenshot below, the node classification-based startup has been selected, and the node type of background processing is selected to ensure that only nodes of this type will be able to run this listener.
- You will also need enough nodes tagged with this node classification to handle the number of threads. Try to limit the threads per node to 6 or less. Thus, if your concurrent listener count total is 24 (as in the above example), you should have 4 nodes dedicated to email processing.
- Try to limit the number of inboxes that you listen to. As the number of inboxes grows, the management of listeners becomes more difficult.
The screenshot below shows a graphical representation of the moving parts in an email listener:
Outbound Email
With outbound email, there are fewer factors to consider within Pega platform. Some of the metrics that need to be understood for outbound email include:
- How many unique email addresses do you want to send email out on?
- Does your email provider have any per-account or global throttles in place that may limit your ability to reach the volume of emails you want to send out in a timely fashion?
- The one item you may need to tune is the number of concurrent TCP connections set on your operating system, which limits the number of connections to your mail service provider. This is often port 25.
Results
With the above detail you should have a better understanding of the factors that go into planning and setting up large-scale email shots, both inbound and outbound.