Best practices for importing data in Pega Sales Automation
Best practices for importing data in Pega Sales Automation
|Description||Describes best practices for importing data in Pega Sales Automation|
|Version as of|
|Capability/Industry Area||Data import|
Moving data from one system to another can be a demanding task considering the challenges involved in importing huge volumes of data for the first time. Another challenge might be to regularly add and update records like leads, contacts, opportunities, and so on.
To simplify the data import process, Pega Sales Automation includes tools to help you to complete your CRM data import faster, more efficiently, and with less effort.
This article describes the recommended approaches for importing data into the Pega Sales Automation application.
Before you can import data, you must review and define answers to the questions listed below, guiding you through the data import process.
- Identifying the selling mode and data model
- Preparing files for import
- Best practices before starting the import
Identifying the selling mode and data model
The Pega Sales Automation data model consists of a set of standard entities that support the sales process. Some of them are:
- Sales goals
For more information about Pega Sales Automation entities, see Pega Sales Automation entities.
Business objects depend on the selling mode that you select for your implementation. In Pega Sales Automation, you can use the following selling modes:
- Business-to-business (B2B) – if you are selling products and services to other businesses.
- Business-to-consumer (B2C) – if you are selling products and services directly to individual consumers.
- Mixed mode – if you are selling products to both individual consumers and to other businesses.
The following diagram shows the Pega Sales Automation business objects that are used in the B2B selling mode and the relationships between these objects:
The following diagram shows the Pega Sales Automation business objects that are used in the B2C selling mode and the relationships between the objects:
The mixed selling mode includes business objects from both B2B and B2C entity diagrams.
For more information, see Pega Sales Automation data model.
Preparing files for import
To import your data into Pega Sales Automation, use comma-separated values (CSV) files. The columns in the CSV files are mapped to fields in the Pega Sales Automation data model by using templates. Use the Pega Sales Automation sample data templates as examples when creating your CSV files.
Sample Contacts.csv file
Note: If you are importing data from other CRM systems like Salesforce & Siebel, Pega Sales Automation provides default templates for importing that data.
Best practices before starting the import
- Review the data types and how entities relate to other entities and their fields.
- If you find that the default templates do not fulfil your requirements, you can extend these templates. For more information, see Data import scenarios.
- Ensure, that the data types are correct and that date formats are unified across all the CSV files.
- Ensure, that the relationships between the records are correct to provide a proper associations between the entities. You can use VLOOKUP in your Excel files to match the parent record IDs with child records in your export CSV files. For example, check if contacts are linked to accounts that they are related to.
- The operator ID in the CSV file must be exactly the same as the created operator ID. Otherwise, the imported work objects associated with the operator ID, for example, organizations, accounts, leads, contacts, or opportunities will not appear in the application list views.
Choosing the best import method
With Pega Sales Automation you can import data in multiple ways. Considering the volume of the data that is getting imported, below are the recommended approaches:
- Import data by using the data import wizard for records < 50K records
- Import data by using file listeners for records > 50K records
The best way to import smaller chunks of data (< 50K records)
Use the Data import wizard if the number of records getting imported is less than 50K records. Data import wizard uses a single thread and has a user interface guiding you through the import process. Users can upload files manually and monitor the import progress.
Consider a scenario of importing organizations of around 50K records into Pega Sales Automation. Below is the sample screenshot of the Organizations csv file to be imported:
Below is the data import mapping screen:
The best way to import large amounts of data (> 50K records)
When you are importing large volumes of data (millions of rows), use data import using File Listener. The data import file listener uses multithreading for faster throughput, while the wizard uses single-thread processing.
Place all your data import files in the folder which was configured in the file listener. When the listener starts from Admin Studio, the importing process runs in the background without any manual intervention.
To configure file listener for data import, see File Listener.
Recommendations for importing around 100 million contacts
Follow the recommended steps for the best performance while using the data import File Listener:
- Before starting the import for all your records, import a few records to start with and fix any issues.
- The size of each csv file used for importing, should not exceed 1 million records.
- Enable configuration templates that are provided out of the box.
- In the App Studio application settings, set the “InitialDataMigration” setting to “true”.
- To support high volumes of data increase number of nodes to depending on time to complete import process. For example, if you are importing 100 million contacts within 10 hours, increase the number of nodes to 10.
- Update the number of threads to around 4 per node. (depends on the system configuration).
- The recommended batch size value for upload is 1000 records.
- To improve performance and to disable creating the audit history, use the Add Only mode for the initial data import.
- To ensure the maximum parallel processing, there must be as many input files for the file listener as there are threads, because each thread processes one file at a time. In Dev Studio, in the Listener properties section, set up the File Listener properties.
Based on the volume of data getting imported, choose the best recommended approach to have an efficient data import.