SPLK-1003 License Management

License Management Detailed Explanation

Splunk’s licensing system ensures that organizations adhere to data ingestion limits and licensing terms while maintaining operational efficiency.

1. License Types in Splunk

Splunk offers several types of licenses, each tailored to specific use cases. Understanding these licenses is crucial for selecting the right one for your environment.

1.1 Enterprise License

• Overview:

  • The Enterprise License is the most comprehensive license, designed for large-scale deployments.
  • It supports all Splunk features, including distributed setups with multiple Indexers, Search Heads, and Forwarders.

• Features:

  • Unlimited Search and Indexing: No functional limits on searches or indexing capabilities, constrained only by the purchased data ingestion limit.
  • Advanced Features:
    • Search Head and Indexer Clustering.
    • Real-time alerts and dashboards.
    • Custom apps and integrations.

• Use Cases:

  • Large enterprises that require robust data ingestion and search capabilities across distributed environments.

1.2 Free License

• Overview:

  • The Free License provides basic Splunk functionality for small-scale or standalone setups.

• Features:

  • Daily Indexing Limit: Typically 500 MB per day.
  • Limited Features:
    • No user authentication (admin access only).
    • No distributed setups (single-instance use only).
    • No clustering support.

• Use Cases:

  • Small businesses or personal projects with minimal data ingestion needs.

1.3 Developer License

• Overview:

  • The Developer License is intended for testing and development purposes.

• Features:

  • Predefined Limits:
    • Limited daily indexing volume (often higher than the Free License).
    • Cannot be used in production environments.
  • Access to all Enterprise features for testing and prototyping.

• Use Cases:

  • Developers and administrators testing Splunk configurations, apps, or integrations.

2. License Pools and Stacks

Splunk organizes license capacity using license pools and license stacks, which provide flexibility and control over license allocation.

2.1 License Pools

• Definition:

  • A license pool is a logical division of the available license capacity, used to allocate specific ingestion limits to different groups or Indexers.

• How it Works:

  • License pools are typically organized by team, department, or use case.
  • Each pool is assigned a daily data ingestion limit.
  • Data from Forwarders and Indexers is counted against the pool’s limit.

• Example:

  • A company has a 100 GB/day license.
  • Two departments (IT and Marketing) use separate pools:
    • IT Pool: 70 GB/day.
    • Marketing Pool: 30 GB/day.

2.2 License Stacks

• Definition:

  • A license stack is the total aggregated license capacity across all pools.
  • The stack represents the overall ingestion capacity purchased by the organization.

• Violations:

  • If the combined ingestion from all pools exceeds the stack capacity, a license violation occurs.
  • Multiple violations in a rolling 30-day period may restrict search functionality.

3. License Management Tasks

Effective license management ensures uninterrupted Splunk operations. Let’s explore the key tasks involved.

3.1 Monitoring License Usage

• Monitoring Console:

  • Access Splunk’s Monitoring Console via Settings > Monitoring Console.
  • Navigate to the License Usage dashboard to view:
    • Daily indexing volume.
    • Pool-specific usage.
    • Historical trends.
  • Example view:
    • “Total usage today: 85 GB” (out of 100 GB available).

• CLI Command:

  • Monitor usage via the command line:

    ./splunk show license-status
    

3.2 Handling License Violations

• What is a License Violation?:

  • A violation occurs when daily ingestion exceeds the assigned license capacity.

• Steps to Resolve:

  1. Identify the Source:

     • Use the License Usage dashboard to locate the source of high ingestion.

  2. Reduce Data Volume:

     • Filter unnecessary data in inputs.conf or props.conf.

     • Example:

       [monitor:///var/log/unnecessary.log]
       disabled = true
       

  3. Upgrade the License:

     • Purchase additional capacity if data ingestion regularly exceeds limits.

• Prevention:

  • Set up alerts for when usage approaches 90% of the daily limit.

3.3 Understanding Daily Indexing Volume

• Definition:

  • The daily indexing volume is the total amount of data (in GB) ingested across all Indexers in a single day.

• Impact on License:

  • Data compression: Splunk applies compression during storage, but licensing is based on raw data size before compression.

• Practical Example:

  • A server sends 2 GB of logs daily.
  • A 10-server deployment will use 20 GB/day of license capacity.

4. Best Practices for License Management

4.1 Set Up Alerts for Usage Limits

• Configure alerts to notify administrators when usage nears capacity:

  1. Navigate to Settings > Searches, Reports, and Alerts.

  2. Create an alert with the following SPL query:

     index=_internal source=*license_usage.log* | stats sum(b) as total_usage | eval total_usage_gb=round(total_usage/1024/1024/1024, 2)
     

  3. Trigger an email alert when total_usage_gb > 90% of the limit.

4.2 Review Indexing Trends Periodically

• Use historical usage data to predict future capacity needs.
• Scale license capacity before data ingestion growth impacts operations.

4.3 Optimize Data Ingestion

• Use props.conf and transforms.conf to filter unnecessary data.

• Apply throttling for high-volume sources:

  [throttle::source::/var/log/bigdata.log]
  interval = 5m
  

Real-World Scenarios

Scenario 1: Managing a Distributed Environment

You have a distributed Splunk deployment with multiple departments. Each department is assigned a specific data ingestion limit.

Steps:

1. Set Up License Pools:

   • IT Team: 50 GB/day.
   • Marketing Team: 30 GB/day.
   • Finance Team: 20 GB/day.

2. Allocate Pools:

   • Assign specific Indexers to each pool based on their usage.
   • Configure the pools in the License Manager.

3. Monitor Usage:

   • Use the Monitoring Console to track each pool’s ingestion:
     • Navigate to Settings > Monitoring Console > License Usage.

Outcome:

You can prevent one department from exceeding its limit and ensure fair resource distribution.

Scenario 2: Handling License Violations

Your Splunk instance reports a license violation due to unexpected data ingestion from a new log source.

Steps:

1. Identify the Issue:

   • Use this SPL query to find the source of high ingestion:

     index=_internal source=*license_usage.log* type=Usage
     | stats sum(b) as bytes by pool, idx
     | eval GB=round(bytes/1024/1024/1024, 2)
     | table pool, idx, GB
     

2. Reduce Data Volume:

   • Adjust inputs.conf to disable or throttle the new source:

     [monitor:///var/log/highvolume.log]
     disabled = true
     

3. Resolve the Violation:

   • Restart the Splunk instance to clear the violation after addressing the root cause.

Outcome:

You mitigate the violation and avoid future penalties by controlling ingestion.

Hands-On Exercises

Exercise 1: Setting Up a License Pool

1. Log in to the License Manager on your Splunk instance.

2. Navigate to Settings > Licensing > Pools.

3. Create a new pool with a specific limit:

   • Name: Team_IT_Pool
   • Limit: 50 GB/day.

4. Assign Indexers to the pool:

   • Add the appropriate Indexers under the Server Assignments section.

5. Monitor the pool usage:

   • Check the License Usage dashboard to ensure the pool operates within limits.

Exercise 2: Creating Alerts for License Usage

1. Create an alert for high license usage:

   • Navigate to Settings > Alerts.

   • Create a new alert and use the following SPL query:

     index=_internal source=*license_usage.log* type=Usage
     | stats sum(b) as bytes by date_mday
     | eval GB=round(bytes/1024/1024/1024, 2)
     | where GB > 90
     

2. Configure the alert:

   • Trigger condition: When result count > 0.
   • Notification: Send an email to the administrator.

3. Test the alert by simulating high usage.

Exercise 3: Optimize License Usage

1. Identify unnecessary data ingestion:

   • Use the License Usage dashboard to find large-volume sources.

   • Example SPL query:

     index=_internal source=*license_usage.log* type=Usage
     | stats sum(b) as total_usage by idx
     | eval GB=round(total_usage/1024/1024/1024, 2)
     

2. Apply filters to reduce data volume:

   • Edit props.conf and transforms.conf:

     [source::/var/log/highvolume.log]
     TRANSFORMS-null = setnull
     
     [setnull]
     REGEX = .
     DEST_KEY = queue
     FORMAT = nullQueue
     

3. Verify the changes by checking the License Usage dashboard.

Advanced Best Practices

Use Splunk Deployment Server

• Deploy consistent configurations to forwarders to prevent unintentional over-ingestion.

Segment Indexers by License Pools

• Assign Indexers handling critical data to a dedicated pool to prioritize their license usage.

Enable Proactive Monitoring

• Use third-party monitoring tools (e.g., Nagios, Grafana) to integrate with Splunk’s licensing metrics for real-time alerts.

Frequently Asked Questions (FAQs)

Q1: What happens during a license violation?

• Impact:
  • Splunk displays warnings in the Web UI.
  • Multiple violations in 30 days restrict search functionality.
• Resolution:
  • Reduce data ingestion and clear violations by restarting Splunk.

Q2: Can I split a license across multiple environments?

• Yes, use license pools to allocate specific capacities to different environments.

Q3: How do I handle a sudden increase in data ingestion?

• Short-term:
  • Filter high-volume sources.
• Long-term:
  • Purchase additional license capacity.

License Management (Additional Content)

Managing licenses effectively is critical in Splunk administration to avoid violations and ensure compliance with indexing limits. This section covers trial licenses, license masters, violation policies, cloud-specific considerations, and file-level details for licensing.

1. Trial License

Overview

A Trial License is a temporary, full-feature Splunk Enterprise license used for evaluation or proof-of-concept (POC) deployments.

Key Characteristics

• Provides complete functionality of Splunk Enterprise:

  • Index clustering

  • Distributed search

  • Authentication integration

  • Role-based access control

• Valid for:

  • 15 or 30 days, depending on the version

• After expiration:

  • Automatically reverts to a Free License with limited capabilities

• Not suitable for production use

Common Use Cases

• Evaluating Splunk before purchase

• Internal lab or demo environments

• POC deployments for clients

Important Note

• Once expired, administrative access may be restricted until a new license is applied or purchased.

2. License Master Role

Definition

In distributed environments, the License Master is the designated Splunk instance responsible for managing license usage and compliance.

Responsibilities of the License Master

• Hosts and distributes license files to connected Splunk instances (called license slaves such as Indexers or Heavy Forwarders)

• Monitors indexing volume across the deployment

• Enforces license pools and daily indexing limits

• Collects logs and metrics related to license usage

Configuration Example (on Indexers)

[license]
master_uri = https://<license-master-host>:8089

This allows the Indexer to forward usage metrics to the License Master.

Management via Splunk Web

• Navigate to:
  Settings > Licensing > License Manager

From here, you can add licenses, view usage statistics, and manage pools and stacks.

3. 30-Day Violation Rule

Definition

Splunk tracks license violations in a rolling 30-day window.

Violation Rule

• You are allowed a maximum of 5 violations in any 30-day period.

• A violation occurs when:

  • The daily indexing volume exceeds the license limit.

Consequence of Violation #6

• On the 6th violation:

  • All non-admin searches are disabled

  • Only admin users can search and resolve the issue

  • Data indexing is not stopped, but alerts and dashboards may fail

Reset Policy

• Violation counts cannot be cleared manually

• The system automatically resets violations as older days roll off the 30-day window

Best Practice

• Set up usage alerts to monitor daily license usage, such as:

  index=_internal source=*license_usage.log* type="Usage"
  | stats sum(b) by pool
  

4. Splunk Cloud Licensing (Overview)

Who It's For

• Organizations using Splunk Cloud, the SaaS version hosted and maintained by Splunk.

Key Differences from On-Prem

Feature	Splunk Cloud	On-Prem
License Pools	Not user-configurable	Fully configurable
Access to .lic Files	No	Yes
License Management	Managed by Splunk	Managed by Admins
Monitoring	Through Cloud Monitoring Console	Via internal dashboards

Monitoring Tools

• Cloud Monitoring Console (CMC)

• Cloud Admin Console

Use Case

• Ideal for enterprises with cloud-first infrastructure or those seeking to offload infrastructure management.

Exam Relevance

• Low to moderate. Expect conceptual questions (e.g., "How is licensing different in Splunk Cloud?").

5. License File Structure and Backup

Location of License Files

• All license files are stored under:

  $SPLUNK_HOME/etc/licenses/
  

• Subdirectories may include:

  • enterprise/

  • forwarder/

  • audit/

File Type and Format

• Splunk licenses use the .lic file extension

• Files are:

  • Digitally signed

  • Tied to a support entitlement or customer license key

Backup Best Practice

• Always back up this directory when:

  • Migrating a License Master

  • Performing disaster recovery or license server failover planning

Command-Line License Installation

./splunk add license /path/to/license-file.lic

This command installs a new license on any Splunk instance with access to the CLI.