PEGACPLSA23V1 Data Model Design

Data Model Design Detailed Explanation

In Pega, Data Model Design refers to defining how data is stored, organized, and accessed within applications. Efficient data models improve application performance, scalability, and maintainability.

5.1 Data Relationships

Data relationships define how data elements interact with one another. By designing relationships correctly, you can ensure that data is reliable, organized, and reusable across applications.

Key Concepts of Data Relationships

1. Single Source of Truth (SSOT)
2. Types of Relationships:
   • One-to-One
   • One-to-Many
   • Many-to-Many

5.1.1 Single Source of Truth (SSOT)

What is Single Source of Truth?

The Single Source of Truth (SSOT) ensures that all data originates from a single, authoritative source. This principle avoids data duplication and ensures consistency.

Why SSOT is Important:

1. Data Consistency: All applications retrieve the same accurate and up-to-date data.
2. Reduced Redundancy: Eliminates duplicate data storage.
3. Simplified Maintenance: Updates to data occur in one place.

Example of Single Source of Truth

Scenario: Customer Data Management

• A bank needs customer information for loan processing, account creation, and support cases.
• Instead of storing duplicate customer data in each process, a single source (CRM system) provides the data.

Implementation:

• Create a Data Page that fetches customer details from the CRM system.
• All applications access this Data Page to retrieve customer data.

5.1.2 Types of Relationships

Data relationships determine how records in one data entity (class) relate to records in another data entity. Pega supports the following types of relationships:

1. One-to-One
2. One-to-Many
3. Many-to-Many

1. One-to-One Relationship

Definition

In a One-to-One relationship, each record in one data class maps to exactly one record in another data class.

Example: Customer and Address

Scenario: Each customer has one primary address.

Customer Table	Address Table
CustomerID: 001	AddressID: A001
Name: John Doe	Street: 123 Elm Street
Email: [email protected]	City: Springfield

• A CustomerID in the Customer Table maps to an AddressID in the Address Table.

Implementation in Pega:

1. Use Properties to define the relationship:
   • Address (Single Page property) stores the address for the customer.
2. Reference the Address Class (e.g., Data-Address) as the source.

2. One-to-Many Relationship

Definition

In a One-to-Many relationship, a single parent record links to multiple child records.

Example: Customer and Orders

Scenario: A customer can place multiple orders.

Customer Table	Order Table
CustomerID: 001	OrderID: O001, CustomerID: 001
Name: John Doe	OrderID: O002, CustomerID: 001
Email: [email protected]	OrderID: O003, CustomerID: 001

• A single CustomerID in the Customer Table maps to multiple orders in the Order Table.

Implementation in Pega:

1. Use Page List Properties to store multiple child records.
   • Example: Orders (Page List) contains all orders for a customer.
2. Reference the child class (e.g., Data-Order) for the Page List property.

3. Many-to-Many Relationship

Definition

In a Many-to-Many relationship, multiple records in one class relate to multiple records in another class. This is typically managed using intermediate tables or joins.

Example: Students and Courses

Scenario: Students enroll in multiple courses, and each course can have multiple students.

Student Table	Course Table	Enrollment Table
StudentID: S001	CourseID: C101	StudentID: S001, CourseID: C101
Name: Alice	CourseName: Math	StudentID: S001, CourseID: C102
		StudentID: S002, CourseID: C101

• The Enrollment Table acts as an intermediate table to connect students and courses.

Implementation in Pega:

1. Create two parent classes: Data-Student and Data-Course.
2. Use a Join Table class (e.g., Data-Enrollment) to store the relationship.
3. Use Page List Properties in the parent classes to reference the join table.

5.1.3 Summary of Data Relationships

Relationship	Definition	Pega Implementation	Example
One-to-One	Each record links to one record in another class.	Single Page Property	Customer → Address
One-to-Many	One parent record links to multiple child records.	Page List Property	Customer → Orders
Many-to-Many	Multiple records in two classes relate to each other.	Join Table and Page Lists	Students ↔ Courses

Best Practices for Data Relationships

1. Follow the Single Source of Truth: Store data in one location and reference it where needed.
2. Avoid Data Duplication: Use relationships instead of duplicating data across classes.
3. Use Page Properties Effectively:
   • Single Page for one-to-one relationships.
   • Page List for one-to-many relationships.
4. Normalize Data: Store data in smaller, related tables to improve performance and scalability.
5. Document Relationships: Maintain clear documentation of data relationships to ensure maintainability.

5.2 Data Classes and Rule Types

In Pega, Data Classes organize data logically, while Rule Types (such as properties, data transforms, and data pages) define, manage, and manipulate the data.

5.2.1 Data Classes

What are Data Classes?

A Data Class is a Pega class used to store, organize, and manage data. It acts as a blueprint for defining data structure and attributes.

Key Characteristics of Data Classes

1. Logical Structure: Defines how data is logically organized.
2. Reusability: Data classes can be shared across multiple case types or applications.
3. Scalability: Data classes are designed to support large datasets efficiently.
4. Integration-Friendly: Data classes can connect to external data sources (e.g., APIs, databases).

Types of Data Classes

1. System-Defined Classes:

   • Predefined Pega classes for out-of-the-box functionality.
   • Example: Data- classes for reusable data structures.

2. Custom Data Classes:

   • User-defined classes to store application-specific data.
   • Example: Data-Customer, Data-Order, or Data-Product.

3. Derived Data Classes:

   • Inherit properties and behavior from a parent class.

Examples of Data Classes

1. Data-Customer: Represents customer information.

   • Properties: CustomerID, Name, Email, Phone.

2. Data-Order: Represents order information.

   • Properties: OrderID, OrderAmount, OrderDate.

3. Data-Address: Represents address details.

   • Properties: Street, City, State, Zip Code.

Steps to Create a Data Class

1. Go to App Studio or Dev Studio.
2. Navigate to Class Explorer → Add a new class under the appropriate parent.
3. Select Data- as the parent class for a custom data class.
4. Define properties for the class (e.g., text, integer, date).

Benefits of Data Classes

1. Reusable: A single data class can be used in multiple case types.
2. Logical Organization: Keeps data structures clean and organized.
3. Supports Inheritance: Allows classes to reuse properties from parent classes.
4. Enables Integration: Simplifies fetching and managing data from external systems.

5.2.2 Rule Types

Pega provides various rule types to define and manage data in a data class.

1. Properties

What are Properties?

Properties define the attributes or fields of a data class. Each property represents a specific type of data, such as text, numbers, or dates.

Types of Properties

1. Single Value: Holds a single piece of data.

   • Example: Name (Text), Age (Integer).

2. Page: Stores a complex data object or single-page structure.

   • Example: Address (Single Page) containing Street, City, State.

3. Page List: Stores a list of complex objects.

   • Example: Orders (Page List) containing multiple orders for a customer.

4. Page Group: Stores a set of pages grouped under a key.

   • Example: Multiple addresses for a customer grouped as Home, Office, etc.

Example: Defining Properties for Data-Customer

Property Name	Type	Description
CustomerID	Integer	Unique identifier for customer
Name	Text	Customer's full name
Email	Text	Customer's email address
Phone	Text	Contact phone number
Address	Single Page	Reference to Address details

2. Data Pages

What are Data Pages?

A Data Page is a rule that retrieves and caches data for use in applications. It acts as an abstraction layer between the application and the data source.

Key Features of Data Pages

1. Scope: Determines who can access the data.

   • Thread: Shared within a single thread (e.g., a single process).
   • Requestor: Shared across the user session.
   • Node: Shared across all users on a server node.

2. Data Loading:

   • Manual Load: Data is loaded programmatically using an activity.
   • Automatic Load: Data loads automatically via connectors, reports, or lookups.

3. Refresh Strategies:

   • Reload if older than: Reload data after a specified interval.
   • Do not reload: Cached data is reused indefinitely.

Example: Customer Data Page

Scenario: Fetch customer data in real time from an external CRM.

1. Data Page Name: D_CustomerData.
2. Scope: Node-level for shared access across users.
3. Source: REST Connector to CRM API.
4. Refresh Strategy: Reload if older than 30 minutes.

3. Data Transforms

What are Data Transforms?

Data Transforms are rules that allow you to map, manipulate, and transform data between structures or classes.

Key Capabilities of Data Transforms

1. Map Data: Map properties between different classes.
2. Set Default Values: Initialize properties with default values.
3. Manipulate Data: Perform calculations, concatenations, and conditional logic.

Example: Mapping Data Between Classes

Scenario: Map customer data from an external API to the Data-Customer class.

Source Property	Target Property	Mapping Logic
external_customer_id	CustomerID	Direct mapping
external_full_name	Name	Direct mapping
external_email	Email	Direct mapping
external_phone_number	Phone	Set default if null

Summary of Data Classes and Rule Types

1. Data Classes: Logical structures for storing and managing data.

   • Example: Data-Customer, Data-Order, Data-Address.

2. Properties: Attributes of data (e.g., Single Value, Page, Page List).

3. Data Pages: Retrieve and cache data from external or internal sources.

4. Data Transforms: Map, manipulate, and initialize data across classes.

5.3 Data Pages

A Data Page is a rule in Pega that retrieves and caches data for use across the application. It provides an abstraction layer for data access, allowing developers to fetch data without worrying about its source or implementation details.

Key Concepts of Data Pages

1. Definition
2. Scopes of Data Pages
3. Loading Data
4. Refresh Strategies
5. Examples
6. Best Practices

5.3.1 Definition of a Data Page

A Data Page is a read-only data structure that fetches and stores data temporarily for use in an application. It can load data from external sources (e.g., REST APIs, databases) or internal sources (e.g., reports, case data).

Why Use Data Pages?

1. Centralized Data Retrieval: Avoid duplicating logic for fetching data.
2. Improved Performance: Use caching to reduce redundant data retrieval operations.
3. Abstraction Layer: Decouple data access logic from application processes.
4. Flexibility: Fetch data from multiple sources without changing the application flow.

5.3.2 Scopes of Data Pages

The scope of a Data Page determines its visibility and lifecycle within the application. Pega provides three scopes:

1. Thread Scope

• Scope: Data is shared within a single thread (e.g., a single case or request).
• Use Case: When data is specific to a single case, task, or flow.
• Lifecycle: The Data Page exists as long as the thread is active.

Example:

• Fetch loan details for a specific loan case.
• The Data Page is visible only within that loan case.

2. Requestor Scope

• Scope: Data is shared across all threads within a single user session (requestor).
• Use Case: When data is relevant for the entire user session but specific to one user.
• Lifecycle: The Data Page exists until the user session ends.

Example:

• Fetch the logged-in user’s profile information (e.g., name, role, permissions).
• The data is available for all cases or flows initiated by the same user during the session.

3. Node Scope

• Scope: Data is shared across all users and threads on a server node.
• Use Case: When data is static or frequently accessed by multiple users.
• Lifecycle: The Data Page exists as long as the node is running or until the refresh criteria is met.

Example:

• Fetch a list of available loan products from a database.
• All users accessing the system can reuse the same data without redundant queries.

Summary of Scopes

Scope	Visibility	Use Case
Thread	Single thread (case/task-specific)	Loan details for a single case
Requestor	Single user session	Logged-in user profile
Node	All users on a server node	Static product catalogs

5.3.3 Loading Data into a Data Page

Data Pages can load data from various sources. The loading mechanism can be manual or automatic.

Loading Mechanisms

1. Manual Load

   • Data is loaded programmatically using an Activity or Data Transform.
   • Example: Fetch data based on user input during runtime.

2. Automatic Load

   • Data loads automatically from a specified source, such as:
     • Connectors: REST, SOAP, or other integration rules.
     • Reports: Fetch data from internal report definitions.
     • Lookups: Retrieve a specific record from a database table.

Example: Automatic Data Page for Customer Data

Scenario: Fetch customer data from an external CRM system.

Configuration	Details
Name	D_CustomerDetails
Scope	Thread
Source	REST Connector to fetch data via API
Refresh Strategy	Reload if older than 30 minutes

Steps to Configure a Data Page

1. Go to Records → Data Model → Data Page.
2. Define the following:
   • Name: D_CustomerDetails.
   • Scope: Thread, Requestor, or Node.
   • Structure: Page (single object) or Page List (list of objects).
3. Select Source: Connector, Report Definition, or Lookup.
4. Define Parameters (e.g., CustomerID) to pass inputs dynamically.
5. Configure Refresh Strategy.

5.3.4 Refresh Strategies

The refresh strategy determines when the Data Page reloads its data. This is important for ensuring data accuracy and performance.

Refresh Options

1. Reload if older than:

   • Reload the data if it is older than a specified time interval.
   • Example: Reload every 30 minutes to ensure fresh data.

2. Do not reload:

   • Use cached data indefinitely without reloading.
   • Example: Static data like a product catalog.

3. Reload per interaction:

   • Always reload data for every interaction.
   • Example: Real-time data like live account balances.

Example: Refresh Strategies

Scenario	Strategy
Fetch customer data (CRM)	Reload if older than 30 minutes
Static product catalog	Do not reload
Live stock price data	Reload per interaction

5.3.5 Examples of Data Pages

1. Static Data (Node Scope):

   • D_LoanProducts: List of loan products fetched from a database.

2. User-Specific Data (Requestor Scope):

   • D_UserProfile: Logged-in user’s profile information.

3. Case-Specific Data (Thread Scope):

   • D_LoanDetails: Loan details for a specific case.

5.3.6 Best Practices for Data Pages

1. Choose the Correct Scope:

   • Use Thread Scope for case-specific data.
   • Use Node Scope for shared static data.

2. Minimize Data Queries:

   • Use caching to reduce frequent data retrieval.

3. Define Clear Refresh Strategies:

   • Set a refresh interval to balance data freshness and performance.

4. Reuse Data Pages:

   • Centralize data access logic by creating reusable Data Pages.

5. Parameterize Data Pages:

   • Pass parameters (e.g., CustomerID) to load specific data dynamically.

6. Monitor Performance:

   • Use Pega Predictive Diagnostic Cloud (PDC) to monitor Data Page performance and optimize as needed.

Summary of Data Pages

1. Definition: A rule that retrieves and caches data for application use.
2. Scopes:
   • Thread: Case-specific.
   • Requestor: User session-specific.
   • Node: Shared across users.
3. Loading Mechanisms:
   • Manual or automatic (via connectors, reports, or lookups).
4. Refresh Strategies: Define when data should reload.
5. Best Practices: Optimize scopes, refresh strategies, and minimize redundant queries.

5.4 Data Integration

Data Integration in Pega allows your application to interact with external systems, ensuring real-time access to accurate and up-to-date data. Integrations are essential for enabling seamless communication between Pega and third-party applications, databases, or APIs.

Key Concepts in Data Integration

1. Integration Options
   • REST, SOAP, and Connect SQL
2. Data Virtualization
3. External System Integration
4. Examples

5.4.1 Integration Options

Pega provides several integration methods to connect to external data sources, such as APIs, databases, and middleware. These options are implemented using connectors.

1. Connectors

What is a Connector?

A Connector is a rule in Pega that allows you to interact with external systems to send, receive, or query data.

Types of Connectors

1. REST Connectors
2. SOAP Connectors
3. Connect SQL

1. REST Connectors

Definition

REST (Representational State Transfer) Connectors integrate with RESTful APIs to exchange data using HTTP methods such as GET, POST, PUT, and DELETE.

When to Use

• Real-time data exchange with modern web services.
• Lightweight and faster communication.

Example: Customer Data Fetch

Scenario: Fetch customer data from an external CRM API.

1. API Endpoint: https://crm.company.com/api/customers/{CustomerID}

2. HTTP Method: GET

3. Request Parameter: CustomerID

4. Response: JSON format:

   {
      "CustomerID": "001",
      "Name": "John Doe",
      "Email": "[email protected]"
   }
   

Steps to Configure a REST Connector

1. Use the Integration Wizard in Dev Studio:

   • Go to Integration → Connectors → Create REST Connector.

2. Provide the following details:

   • Endpoint URL: URL of the REST API.
   • HTTP Method: GET, POST, PUT, DELETE.
   • Parameters: Inputs such as CustomerID.

3. Define the Data Page to use the REST Connector as its source.

4. Map the API response to Pega properties using a Data Transform.

2. SOAP Connectors

Definition

SOAP (Simple Object Access Protocol) Connectors integrate with older, XML-based web services.

When to Use

• Integrations with legacy systems using WSDL (Web Service Definition Language).
• Complex operations requiring structured data exchanges.

Example: Insurance Policy Service

Scenario: Retrieve an insurance policy from a SOAP web service.

1. WSDL File: Provided by the external system.

2. Request: Input policy ID.

3. Response: XML format:

   <Policy>
      <PolicyID>12345</PolicyID>
      <CustomerName>John Doe</CustomerName>
      <Premium>500</Premium>
   </Policy>
   

Steps to Configure a SOAP Connector

1. Use the Integration Wizard to import the WSDL file.
2. Configure request and response mappings.
3. Test the connection using SOAP UI or Pega tools.

3. Connect SQL

Definition

Connect SQL allows Pega to execute SQL queries on relational databases for data integration.

When to Use

• Direct interaction with relational databases (e.g., Oracle, MySQL).
• Fetch or update large volumes of data efficiently.

Example: Fetch Customer Orders

Scenario: Retrieve all orders for a specific customer from a relational database.

SQL Query
SELECT * FROM Orders WHERE CustomerID = '001';

Steps to Configure Connect SQL

1. Create a Database Table rule to connect to the external database.
2. Use Connect SQL to write SQL queries.
3. Map query results to Pega properties.
4. Use the Connect SQL rule as the source for a Data Page.

5.4.2 Data Virtualization

What is Data Virtualization?

Data Virtualization allows Pega to fetch data in real time from external systems without storing it locally in the Pega database. This avoids redundant data storage and ensures the most up-to-date information is always retrieved.

When to Use Data Virtualization

1. When data is frequently updated in an external system.
2. To avoid duplicating large datasets locally.

Example: Real-Time Customer Information

• Fetch customer details directly from an external CRM system using a REST Connector.
• Use a Node-Level Data Page to cache the data and refresh periodically.

Benefits of Data Virtualization

1. Reduces data redundancy and storage costs.
2. Ensures data consistency by fetching real-time updates.
3. Improves performance by caching frequently accessed data.

5.4.3 External System Integration

Pega integrates with external systems via:

1. Middleware Tools:

   • Kafka, MuleSoft, or IBM MQ for enterprise integrations.

2. APIs:

   • REST and SOAP APIs to interact with systems like CRM, ERP, and payment gateways.

3. Databases:

   • Direct database connections using Connect SQL.

Examples of External Integrations

1. Fetch Customer Data: Use a REST Connector to retrieve customer data from Salesforce.
2. ERP Integration: Use a SOAP Connector to fetch inventory levels from SAP.
3. Payment Processing: Send payment requests to Stripe or PayPal using a REST API.

5.4.4 Summary of Data Integration

Integration Method	Use Case	Key Tool
REST Connectors	Integrate with RESTful APIs	REST Connector Rule
SOAP Connectors	Integrate with legacy systems	SOAP Connector Rule
Connect SQL	Interact with relational databases	Connect SQL Rule
Middleware (Kafka, MuleSoft)	Enterprise integrations	Integration Middleware
Data Virtualization	Real-time external data access	Data Pages + Connectors

Best Practices for Data Integration

1. Centralize Data Access: Use Data Pages to fetch and cache external data.
2. Avoid Redundant Data: Implement data virtualization instead of duplicating data locally.
3. Monitor Integrations: Use Pega Predictive Diagnostic Cloud (PDC) to monitor connector performance.
4. Handle Errors Gracefully: Implement retry mechanisms and error handling for failed integrations.
5. Secure Integrations: Use encryption and secure authentication (e.g., OAuth) for external API connections.
6. Test Integrations Thoroughly: Use tools like Postman or SOAP UI to validate connector behavior.

5.5 Data Validation

Data Validation in Pega ensures that data entered by users or retrieved from external sources meets specific rules, constraints, and formats. Proper validation prevents errors, enhances data quality, and ensures consistency across the application.

Key Topics in Data Validation

1. Validation Rules
2. Edit Validate and Edit Input Rules
3. Declarative Constraints
4. Examples
5. Best Practices

5.5.1 Validation Rules

What are Validation Rules?

A Validation Rule in Pega defines constraints or conditions that data must meet. These rules are applied to ensure user inputs or external data are valid before progressing in the case flow.

Types of Validation

1. Field-Level Validation:

   • Ensures data entered into a field meets the expected format or condition.
   • Example: An email field must contain @ and a domain name.

2. Form-Level Validation:

   • Validates multiple fields within a form before submission.
   • Example: Ensure required fields like Name and Phone are not empty.

3. Business Logic Validation:

   • Validates data against business rules or complex conditions.
   • Example: Loan Amount must be less than the customer’s annual income.

Example of a Field-Level Validation

Scenario: Validate that the "Email Address" field contains a valid email format.

Property	Validation	Message
Email	Must match pattern: *@*.com	"Please enter a valid email."

Steps to Configure:

1. Create a Validation Rule.
2. Add a Field Validation Condition: Use an expression like Email contains @ and ends with .com.
3. Define the Error Message.

Example of Form-Level Validation

Scenario: Validate that all required fields in a loan application form are filled.

Field	Validation	Message
Name	Required	"Name cannot be empty."
Loan Amount	Required	"Loan Amount is required."
Phone Number	Must be numeric	"Phone Number must be valid."

Steps to Configure:

1. Add a Validation Rule to the form.
2. Use Required Fields and conditions in the rule definition.

5.5.2 Edit Validate and Edit Input Rules

1. Edit Validate Rules

Definition:
An Edit Validate rule checks user-entered data against a specific pattern or condition during input.

When to Use Edit Validate Rules

• To validate formats for fields like email, phone numbers, or ZIP codes.
• To enforce input conditions during user interactions.

Example: Validate Phone Number Format

Scenario: Ensure the phone number contains exactly 10 digits.

1. Create an Edit Validate Rule:
   • Use a regex pattern: [0-9]{10}.
2. Attach the rule to the Phone Number property.

Result: If the user enters a value other than 10 digits, the system displays an error message.

2. Edit Input Rules

Definition:
An Edit Input rule automatically reformats user input into a standardized format.

When to Use Edit Input Rules

• To clean or format user-entered data before saving it.
• Example: Converting phone numbers into a consistent format (e.g., (123)-456-7890).

Example: Format Phone Number

Scenario: Convert the entered phone number 1234567890 to the format (123)-456-7890.

1. Create an Edit Input Rule:

   • Logic to reformat the input:

     return "(" + input.substring(0,3) + ")-" + input.substring(3,6) + "-" + input.substring(6);
     

2. Attach the rule to the Phone Number property.

5.5.3 Declarative Constraints

What are Declarative Constraints?

Declarative Constraints in Pega ensure that data meets specific conditions or business rules automatically without requiring manual checks.

Key Features

1. Automatic: Constraints are triggered whenever relevant properties are updated.
2. Reusable: Define a constraint once and reuse it across multiple workflows.
3. Error Handling: Display custom error messages when constraints are violated.

Example: Validate Loan Amount

Scenario: Ensure the Loan Amount does not exceed the Annual Income.

Condition	Message
Loan Amount > Annual Income	"Loan Amount cannot exceed income."

Steps to Configure:

1. Go to Records → Decision → Constraint Rule.
2. Define the condition: LoanAmount > AnnualIncome.
3. Add an Error Message for constraint violation.

5.5.4 Examples of Data Validation

1. Email Field Validation:

   • Edit Validate Rule: Ensure email contains @ and .com.

2. Numeric Validation:

   • Field Validation: Phone Number must contain only digits.

3. Business Rule Validation:

   • Declarative Constraint: Loan Amount ≤ Annual Income.

4. Data Transformation:

   • Edit Input Rule: Format phone numbers as (123)-456-7890.

5. Form-Level Validation:

   • Validate all required fields in a loan application form.

5.5.5 Best Practices for Data Validation

1. Centralize Validation Logic: Use Validation Rules and Edit Validate Rules to centralize checks.
2. Use Declarative Constraints: Automatically enforce business rules whenever relevant properties are updated.
3. Format Inputs with Edit Input: Standardize user input formats for consistency.
4. Validate at the Field Level: Use property validations for critical fields like email, phone, and SSNs.
5. Avoid Redundancy: Do not repeat validation logic; reuse rules wherever possible.
6. Provide Clear Error Messages: Ensure error messages are user-friendly and descriptive.
7. Test Thoroughly: Validate inputs with edge cases (e.g., invalid formats, empty fields).

Summary of Data Validation

1. Validation Rules: Define constraints for properties at field or form level.
2. Edit Validate Rules: Validate data input against specific formats or patterns.
3. Edit Input Rules: Reformat user inputs into a standardized format.
4. Declarative Constraints: Automatically enforce conditions on data.
5. Examples:
   • Validate phone number formats.
   • Ensure Loan Amount ≤ Annual Income.
   • Format phone numbers consistently.

5.6 Performance Optimization

Performance optimization focuses on improving data retrieval, reducing redundant processing, and ensuring efficient database interactions. Proper optimization techniques enhance scalability, responsiveness, and user experience.

Key Topics in Performance Optimization

1. Efficient Data Retrieval
2. Caching Strategies
3. Avoiding Redundancy
4. Database Indexing
5. Examples
6. Best Practices

5.6.1 Efficient Data Retrieval

Efficient data retrieval minimizes the number of queries to the database or external systems, thereby reducing response times and improving application performance.

Techniques for Efficient Data Retrieval

1. Avoid Unnecessary Queries:

   • Use Data Pages to cache data instead of querying the database repeatedly.
   • Example: Use a Node-Level Data Page to store static reference data like product catalogs.

2. Pagination:

   • Retrieve and display large datasets in smaller chunks using pagination.
   • Example: Fetch 20 records at a time instead of all records at once.

3. Optimize Report Definitions:

   • Use Report Definitions instead of custom SQL queries where possible.
   • Only fetch the required columns and apply filters to minimize data retrieval.

4. Lazy Loading:

   • Load data on demand instead of preloading it.
   • Example: Fetch detailed customer data only when the user clicks on the “View Details” button.

5. Use Declarative Data:

   • Leverage Declarative Expressions to calculate property values automatically without redundant queries.

Example: Optimize Report Definition for Customer Orders

Scenario: Display orders for a customer in a dashboard.

Before Optimization	After Optimization
Fetch all columns and all orders at once.	Fetch only required columns (OrderID, Date, Total) and filter by CustomerID.

Optimized Query:

• Use a Report Definition with filters:

  SELECT OrderID, OrderDate, OrderTotal  
  FROM Orders  
  WHERE CustomerID = '001'  
  ORDER BY OrderDate DESC  
  LIMIT 20;  
  

5.6.2 Caching Strategies

Caching improves performance by reducing the need for frequent data retrieval from databases or external systems.

Types of Caching in Data Pages

1. Thread-Level Cache:

   • Data is cached within a single process/thread.
   • Use Case: Case-specific or short-lived data (e.g., loan details for a single application).

2. Requestor-Level Cache:

   • Data is cached for the duration of the user session.
   • Use Case: User-specific data (e.g., logged-in user’s profile).

3. Node-Level Cache:

   • Data is cached at the server node level and shared across users.
   • Use Case: Static or frequently accessed data (e.g., product catalogs, exchange rates).

Refresh Strategies for Caching

1. Reload if older than X minutes:

   • Reload data after a specified interval.
   • Example: Refresh product catalogs every 2 hours.

2. Do not reload:

   • Use cached data indefinitely.
   • Example: Static reference data that rarely changes.

3. Reload per interaction:

   • Reload data every time it is accessed.
   • Example: Real-time stock prices.

Example: Cache Static Product Data

Scenario: Fetch product details from a database and cache them for 1 hour.

1. Data Page Configuration:
   • Scope: Node
   • Refresh Strategy: Reload if older than 1 hour.

Benefit: The product data is fetched once and shared across all users, reducing database queries.

5.6.3 Avoiding Redundancy

Avoid storing duplicate or unnecessary data, as it leads to performance bottlenecks and inconsistency.

Techniques to Avoid Redundancy

1. Normalization:

   • Break data into logical tables to avoid duplication.
   • Example: Store customer and address details in separate tables and link them using keys.

2. Use Data Virtualization:

   • Fetch real-time data from external systems instead of storing it locally.

3. Reference Data Pages:

   • Centralize data access using Node-Level Data Pages for reusable and shared data.

5.6.4 Database Indexing

Database indexing improves query performance by enabling faster searches and retrieval of data.

What is Indexing?

An index is a database structure that allows faster lookup of specific rows in a table.

When to Use Indexing

1. For frequently queried columns.
2. For columns used in WHERE clauses, filters, or sorting.
3. For foreign keys and primary keys.

Example: Indexing for Customer Lookup

Scenario: Frequently query the CustomerID field in the Customer table.

1. Add an index on the CustomerID column:

   CREATE INDEX idx_customer_id ON Customer (CustomerID);  
   

2. Result: Queries using WHERE CustomerID = '001' will run faster.

Benefits of Indexing

1. Reduces query response times.
2. Improves application performance for data-intensive operations.
3. Enhances scalability as data volume grows.

5.6.5 Examples of Performance Optimization

1. Efficient Queries: Optimize Report Definitions to fetch only required data.
2. Node-Level Cache: Cache frequently accessed product catalogs.
3. Lazy Loading: Load detailed customer data only when needed.
4. Indexing: Add indexes to frequently queried fields like CustomerID.
5. Avoid Redundancy: Use normalized data structures and reference Data Pages.

5.6.6 Best Practices for Performance Optimization

1. Use Node-Level Data Pages: Cache static or frequently used data to reduce queries.
2. Optimize Queries: Fetch only required columns and apply filters efficiently.
3. Implement Pagination: Load large datasets in smaller chunks.
4. Leverage Indexing: Add indexes on frequently filtered or sorted database fields.
5. Reduce Redundancy: Follow normalization principles and use virtualized data.
6. Monitor Performance: Use tools like Pega Predictive Diagnostic Cloud (PDC) to identify and resolve performance bottlenecks.
7. Minimize External Calls: Avoid unnecessary API or database calls by caching results.
8. Test with Realistic Data: Simulate production data volumes to evaluate performance.

Summary of Performance Optimization

1. Efficient Data Retrieval: Minimize queries, use lazy loading, and optimize reports.
2. Caching Strategies: Use Thread, Requestor, or Node-level caches for different use cases.
3. Avoid Redundancy: Normalize data and use virtualization for real-time access.
4. Database Indexing: Add indexes to improve query performance.
5. Best Practices: Optimize queries, leverage caching, and monitor system performance.

Conclusion of Data Model Design

We have now covered all the key topics in Data Model Design:

1. Data Relationships: One-to-One, One-to-Many, and Many-to-Many relationships.
2. Data Classes and Rule Types: Organize data with properties, transforms, and Data Pages.
3. Data Pages: Centralize and cache data for efficient access.
4. Data Integration: Connect external systems via REST, SOAP, and SQL.
5. Data Validation: Ensure data integrity with validation rules, Edit Validate, and constraints.
6. Performance Optimization: Retrieve data efficiently, use caching, and leverage indexing.

Data Model Design (Additional Content)

1. Data Relationships – Advanced Join Table Reuse

Many-to-Many with Keyed Page Group for Reusability

In complex many-to-many relationships, a Join Table class (e.g., Data-Enrollment) is commonly used to relate two entities (e.g., Students and Courses). To make this relationship more reusable and extensible, consider modeling the association using a Keyed Page Group.

Why Use Keyed Page Group:

• Allows the system to index relationships using a meaningful key, such as Course ID or Student ID.

• Supports easy access and iteration over specific related records.

• Enhances clarity and reuse across multiple case types or modules.

Example:

Property: Enrollments (Page Group)
Class: Data-Enrollment
Key: CourseID

You can now access enrollment details using:
.Enrollments(C101).Status, .Enrollments(C102).Grade, etc.

This design is particularly powerful when you need structured, indexed access to relationship data in reports, UI sections, or integrations.

2. Data Classes and Rule Types – Enhanced Usage

a. Reusing Data Classes Across Case Types

To avoid duplication and promote reuse, logic and properties encapsulated in Data Classes should be shared across multiple Case Types using Page or Page List references.

Best Practice for Reuse:

1. Create a reusable Data Class, e.g., Data-Product.

2. In your Case Type, define a Single Page or Page List:

• .ProductDetails (Page) → Data-Product

• .ProductList (Page List) → Data-Product

3. Reference this page across steps, UI forms, validations, and transforms.

This enables centralized updates and ensures data standardization across the application.

b. Data Transform – Iterating a Page List

A common real-world pattern involves looping through a Page List to process or aggregate values.

Example Use Case: Sum total value of all order items.

Loop: OrderItems()
    .TotalAmount += OrderItems().ItemAmount

This logic in a Data Transform accumulates item values into a single .TotalAmount property — often used in invoicing or summary screens.

3. Data Pages – Structural Clarification and Parameterization

a. Page vs Page List Data Pages

Pega supports two structures for Data Pages:

Structure	Description	Example
Page	Holds a single object	D_CustomerDetails for one customer
Page List	Holds multiple objects	D_AllLoanTypes returning all loan types

This is frequently tested in exams — especially in scope + structure pairings like:

• Node + Page List → Static reference data

• Thread + Page → Case-specific dynamic fetch

b. Parameterized Data Pages – Use Case

Use parameters in Data Pages to fetch context-specific data.

Example:

D_CustomerDetails[CustomerID: "C1001"]

This pattern supports:

• Real-time queries to external systems

• Dynamic data fetches for form population

• Avoids hardcoding, promotes reuse

4. Data Integration – Error Handling Strategy

When integrations (e.g., REST or SOAP connectors) fail, it is critical to handle them gracefully and predictably.

Best Practice: Use Error Handling Flows

• Configure Flow-level error handling via Error Paths or Alternate Stages.

• Use StepStatusFail in activities or HasMessages condition in Data Transforms to detect failures.

• Redirect users to:

  • A “Retry or Contact Support” screen

  • A manual processing queue via assignment

This approach aligns with robust enterprise-grade resilience design, and is a common exam scenario.

5. Data Validation – Complex Conditions and Declarative Constraints

a. Multi-Field Validation Scenario

Sometimes, validation requires composite logic: e.g., "Field A and Field B must be filled together, or both left blank."

Declarative Expression or Validate Rule Logic:

IF (IsMissing(.A) AND NOT IsMissing(.B)) OR (NOT IsMissing(.A) AND IsMissing(.B))
THEN Show Error: "Please fill both A and B or leave both empty."

b. Declarative Constraint – Under the Hood

Declarative Constraints are implemented via:

• A When rule (evaluating the condition)

• Bound to a property through a Constraint rule

• Upon violation, Pega automatically displays an error message

This is reactive validation, triggered whenever data changes — and commonly appears in rule resolution and form validation questions.

6. Performance Optimization – Monitoring & Indexing Practices

a. Monitoring with Clipboard Analyzer & PDC

Use tools like:

• Clipboard Analyzer: To inspect whether Data Pages are overused, stale, or large.

• Pega PDC (Predictive Diagnostic Cloud): To detect high-frequency queries, slow report execution, or repeated page reloads.

These tools support data-driven optimization of performance.

b. Index Usage in Report Definitions

Exam questions often focus on how to design efficient reports. Emphasize:

• Filter and sort by indexed properties (e.g., .SubmissionDate, .Status)

• Avoid filters on non-indexed computed properties

• Use database views or pre-aggregated tables when joining multiple large tables

Failing to do so leads to full table scans — a common performance bottleneck.

7. Data Reuse and Governance – Enterprise-Level Practices

Why Governance Matters

Strong data modeling governance ensures:

• Avoidance of rule duplication

• Modular and maintainable data structures

• Compliance with standards and enterprise architecture policies

Best Practices and Tools:

1. Use built-on applications to centralize reusable Data- classes.

2. Organize reusable assets in shared RuleSets (e.g., Common-Data, Enterprise-Rules).

3. Enforce naming conventions:

• Page names: CustomerInfo, ProductList

• Property names: CamelCase or enterprise-standard styles

4. Leverage Application Quality Metrics in Dev Studio to audit violations or unused rules.

LSA Perspective:

LSA exam scenarios often test design consistency and governance, especially in modular application setups and reuse strategies across business lines.