SPLK-3001 Custom Add-ons

Custom Add-ons Detailed Explanation

1. What Are Custom Add-ons in Splunk ES?

Splunk ES relies on data that is formatted and structured according to the Common Information Model (CIM). This formatting is typically handled by Technology Add-ons (TAs), which:

• Extract fields from raw data

• Normalize those fields

• Tag events with appropriate CIM categories

However, not all vendors provide official TAs. In such cases, Custom Add-ons must be developed.

2. Key Tasks in Building Custom Add-ons

Custom Add-ons are primarily built using the Splunk Add-on Builder App, which is available from Splunkbase. This tool provides a step-by-step interface to guide you through building a CIM-compliant TA.

a. Create Data Inputs

• Define data ingestion methods for your source:

  • Syslog

  • REST API

  • File-based ingestion

Example: If integrating a custom security tool that offers a REST API, you can build a modular input to poll data regularly and ingest it into Splunk.

b. Define Field Extractions

• Use regular expressions or delimiters to extract fields like:

  • src, dest, user, signature, severity

• These fields should follow CIM naming conventions.

Poor field extraction leads to:

• Missing dashboard values

• Failed correlation searches

c. Create Event Types and Tags

Event types categorize logs, while tags map them to CIM models.

Examples:

• Event type: vpn_auth_logs

• Tags: tag=authentication, tag=network

These allow ES to recognize what kind of data it's dealing with.

d. Map to CIM Data Models

The final and most critical step is linking your fields to the correct CIM data model, such as:

• Authentication

• Intrusion Detection

• Malware

• Change Analysis

This ensures your data is usable across correlation searches, risk rules, and dashboards.

3. CIM Compliance

CIM compliance is not just about field presence—it’s also about naming, typing, and tagging consistency.

a. Naming Conventions

Follow CIM-standard field names exactly:

• Use src, not src_ip

• Use user, not username

Field aliasing may help in certain cases, but ideally, your extractions should match CIM from the start.

b. Required Tags

Apply relevant tags that identify the function of the event. For example:

• tag=authentication

• tag=update

• tag=malware

Tags help link your events to data models and are essential for visibility within ES.

c. Validate Field Types and Formats

CIM expects certain fields to be of specific data types or value formats.

Example:

• src and dest should be IP addresses.

• user should be a string identifier.

• severity should be low/medium/high or a numeric scale.

4. Packaging and Deployment

After building and testing, custom add-ons must be packaged and deployed properly.

a. Packaging Guidelines

• A TA (Technology Add-on) should not include dashboards or UI elements.

• Only include:

  • Inputs

  • Props/transforms

  • Field extractions

  • Tags and event types

This keeps the TA lightweight and focused on data handling, not visualization.

b. Deployment Locations

• Universal Forwarder: For simple log forwarding (no parsing)

• Heavy Forwarder: For local parsing and field extractions

The TA must be deployed where field extraction logic is applied, usually at index time on heavy forwarders or at search time on search heads (depending on configuration).

5. Testing and Validation

Once the TA is deployed, validation is essential to ensure it works as expected in the Splunk ES environment.

a. CIM Validation Dashboard

• Found in the Splunk Add-on Builder.

• Lets you run sample events against your TA to verify:

  • Field extractions

  • Tag application

  • Model population

b. Data Model Audit Tool

• Found in Splunk ES under the Data Model Audit Dashboard.

• Helps track how many events from your TA are appearing in the expected data models.

• Identifies gaps in tagging or extraction.

Regular testing ensures long-term reliability and compatibility with ES content.

6. Summary of Custom Add-ons

Task Area	Key Details
Add-on Creation	Use Add-on Builder to define inputs, extractions, tags, and CIM mappings
CIM Compliance	Match field names, types, and tags exactly to data model expectations
Packaging & Deployment	Package as a TA without UI; deploy to forwarders or parsing layer
Testing Tools	Use CIM Validation and Data Model Audit dashboards to confirm effectiveness

Custom Add-ons (Additional Content)

1. Sample Directory Structure for a Technology Add-on (TA)

When creating a custom TA for use with Splunk Enterprise Security, it's important to follow a standardized directory structure. This ensures compatibility with Splunk's deployment tools, and proper functioning across forwarders, indexers, and search heads.

Example: TA for a Custom Firewall Integration

TA-custom_firewall/
├── default/
│   ├── props.conf            # Field extraction, sourcetype rules
│   └── transforms.conf       # Regex-based field rewrites, extractions, lookups
├── local/                    # Local overrides (not typically packaged for deployment)
├── bin/                      # Custom scripts for modular inputs or external lookups (optional)
├── metadata/
│   └── default.meta          # Permissions and visibility for knowledge objects
└── README                    # Documentation and versioning notes

Key Notes:

• default/: Contains the base configuration files that define core TA behavior. These are deployed consistently across environments.

• local/: Contains environment-specific overrides (should not be included when packaging for production).

• bin/: Stores Python or shell scripts used for data inputs or external enrichment, if applicable.

• metadata/: Sets permissions and object sharing (e.g., global visibility of event types and tags).

Having this structure ensures the TA is deployable via deployment servers, cleanly packaged, and CIM-compatible for use with Splunk ES.

2. Manual Review of Auto-Generated Config Files

When using the Splunk Add-on Builder (a GUI tool to assist in TA creation), it’s important to remember:

Not all generated configurations are production-ready.

a. Common Issues That Require Manual Review:

• props.conf inaccuracies:

  • Missing REPORT- stanzas for field extractions

  • Incorrect sourcetype-to-field mappings

  • Missing timestamp extraction (TIME_FORMAT, MAX_TIMESTAMP_LOOKAHEAD)

• transforms.conf logic errors:

  • Faulty regex

  • Unused stanza references

  • Misaligned field aliases or calculated fields

b. Best Practice:

Always manually audit default/props.conf and transforms.conf before deployment, even if using Add-on Builder.

This step ensures that:

• Fields conform to CIM field naming standards (e.g., src, dest, user)

• Tags and event types are correctly defined

• Log data will populate the expected data models in Splunk ES

Summary of Enhancements

Topic	Enhancement Description
TA Directory Structure	Provides a best-practice folder layout for production-ready add-ons
Add-on Builder Manual Review	Warns that auto-generated config files may not fully meet ES/CIM requirements