SPLK-3001 Lookups and Identity Management

Lookups and Identity Management Detailed Explanation

1. What Are Lookups in Splunk ES?

Lookups are external reference tables used by Splunk to map field values to additional information. In Splunk ES, lookups are essential for adding business, geographical, or organizational context to events.

Rather than relying on raw data alone (like an IP address or username), lookups help answer questions like:

• Who does this user belong to?

• What type of device is this IP associated with?

• What is the business function of this system?

2. Types of Lookups

a. CSV File Lookups

• These are static files stored locally within Splunk.

• Commonly used for simple field-to-value mappings.

• Format: Comma-separated values (CSV), e.g.,:

  ip,country
  192.168.1.1,United States
  

• Use Cases:

  • Map IP addresses to countries.

  • Link usernames to roles or departments.

  • Tag hosts with asset types (e.g., laptop, server, firewall).

b. KV Store Lookups

• Key-Value store lookups are dynamic and editable via the Splunk Web UI or API.

• More scalable and interactive than CSVs.

• Support add/update/delete operations through scripts or the GUI.

• Use Cases:

  • Track live inventory of assets.

  • Update threat intelligence indicators.

  • Maintain dynamic user role mappings.

c. External Lookups

• These use custom scripts (e.g., Python) to enrich data in real-time during a search.

• Can query external databases or APIs (like VirusTotal or GeoIP).

• Use Cases:

  • Enrich IPs with live threat feed data.

  • Check user risk profiles from third-party identity platforms.

  • Query live vulnerability databases to validate host risk levels.

3. Use Cases for Lookups in Splunk ES

Lookups enhance many critical areas of Splunk ES, including correlation logic, dashboards, and investigation tools.

Example Applications:

• Map IP addresses to asset hostnames or device types.

• Link usernames to departments, business roles, or access levels.

• Add custom business context (e.g., asset criticality, geographic region).

• Filter or enrich correlation searches by appending data from lookup tables.

This allows alerts to be prioritized based on who or what is involved, not just what occurred.

4. Identity Management in Splunk ES

This is a specific application of lookups, focused on understanding “who” (identities) and “what” (assets) are part of any given event.

a. Assets and Identities Lookup Files

• Found in:

  $SPLUNK_HOME/etc/apps/SA-IdentityManagement/lookups
  

• Key Files:

  • assets.csv: Maps IPs/MACs to hostnames, asset type, location, owner, and priority.

  • identities.csv: Maps usernames or email addresses to full names, departments, and access tiers.

These files provide Splunk ES with enriched identity context used in:

• Dashboards

• Correlation searches

• Risk calculations

b. The Priority Field

• Assets and identities include a priority field:

  • Values can be: Low, Medium, High, or Critical.

  • This field directly affects risk scoring in correlation rules and alerts.

Example:

A “Critical” asset involved in an event will cause the alert to have a higher risk score than if it involved a “Low” priority system.

This ensures that important systems/users get the most attention from analysts.

c. Management and Validation

• Splunk ES provides the Asset and Identity Center dashboard to:

  • View loaded identity and asset records.

  • Check field mappings and record quality.

  • Validate lookup status and usage across correlation searches.

This dashboard is essential for ongoing maintenance of identity context and ensuring that field mappings are accurate.

5. Summary of Lookups and Identity Management

Element	Description
CSV Lookups	Static tables for mapping values like IP-to-location or user-to-role
KV Store Lookups	Dynamic lookups editable via UI/API for real-time enrichment
External Lookups	Script-based real-time context from external sources or APIs
Identity Management Files	assets.csv and identities.csv provide context for “who” and “what”
Priority Field	Defines importance of assets/users and affects risk scoring
Validation Dashboard	Asset and Identity Center used to validate, view, and manage lookup data

Lookups and Identity Management (Additional Content)

1. Using Automatic Lookups: props.conf + transforms.conf

Automatic lookups are a core part of making data enrichment seamless in Splunk. They allow Splunk to append values from a lookup table to events at search time, without requiring the user to explicitly call the lookup command.

a. How It Works

• In transforms.conf, you define the lookup logic (i.e., the file to use and how fields map).

• In props.conf, you define when and for which sourcetype that lookup should be applied.

b. Example Configuration

transforms.conf

[add_department_info]
filename = identities.csv
match_type = WILDCARD(user)
lookup_fields = user
output_fields = department, role

props.conf

[sourcetype=syslog]
LOOKUP-add_department = add_department_info user OUTPUT department, role

This will enrich events from sourcetype=syslog by matching the user field with values in identities.csv, automatically appending the user’s department and role.

2. Handling Lookup Failures (Fallback Behavior)

When a lookup fails (i.e., the input field has no match in the lookup table), Splunk:

• Does not generate an error

• Returns null (empty) values for the output fields

• The event continues processing as normal

Example:

If a user value in an event is john.doe, but john.doe does not exist in identities.csv, then:

user="john.doe", department="", role=""

This can be handled in SPL using logic like:

... | eval user_context=if(isnull(department), "unknown", department)

Or filtered explicitly:

... | where isnotnull(department)

This behavior ensures query robustness, even when lookups are incomplete or in-progress.

3. Using the Splunk Web GUI: Lookup Editor for KV Store Lookups

Splunk Enterprise Security supports KV Store-based lookups, which are ideal for storing dynamic identity or asset context.

a. Editing KV Store Data via the GUI

If the Lookup Editor app is installed (available via Splunkbase), you can manage KV Store collections directly from the UI:

• Navigate to:
  Settings > Lookups > Lookup Editor

• From here, you can:

  • View all KV Store-based lookups

  • Add, delete, or update rows (e.g., identities, asset mappings)

  • Edit individual fields (e.g., priority, owner, location)

  • Sort and search within the lookup table

b. Common KV Store Lookup Targets in Splunk ES:

KV Store Lookup	Purpose
identities.csv	Maps user → real_name, department
assets.csv	Maps ip or mac → asset_type, site
Custom collections	User-created lookups for threat feeds or roles

Using the GUI editor reduces the need for CLI interaction and is suitable for analysts or administrators who need to manage records frequently.

Summary of Enhancements

Topic	Enhancement Description
Automatic Lookups	Shows how to apply lookup enrichment via props.conf and transforms.conf
Lookup Failure Behavior	Explains how Splunk handles missing matches by returning nulls, not errors
GUI-Based KV Store Management	Introduces the Lookup Editor for easy creation/editing of dynamic lookup entries