Skip to main content

Upgrade Distributed Splunk Environment

Follow the steps below to upgrade the UXM monitoring solution On-Premise, the software can be downloaded from here: Download server software. (Splunk app downloads will be moved to Splunkbase in 2026)

warning

Please note that older uxmapp installations calls app and indexes for mcg_uxm, newer installations uses uxmapp_ as prefix for app and indexes

Upgrade Splunk​

Ensure Splunk are upgraded to newest version on both Search Heads, Indexers and the Heavy Forwarder where UXM is installed, Splunk 9.2 and newer are supported by UXM.

Upgrade Indexer​

No upgrades needed.

Upgrade Search Head​

  1. Update with new apps

    Goto Apps -> Manage Apps and upload the uxmapp_sh_YYYY.MM.DD.tar.gz or mcg_uxm_distributed_sh_onpremise_YYYY.MM.DD.tar.gz app file depending on app installed on you system.

    Remember to click "Upgrade app checkbox" and click Upload, restart Splunk later when asked.

    info

    You can continue if error "Error connecting to /services/apps/local: The read operation timed out" occurs, normally app is uploaded and old app overridden successfully.

    Press restart later if asked.

    Upload new uxmapp_waterfall and uxmapp_worldmap visualization apps if released and restart Splunk later.

  2. Update permissions for uxm roles

    For uxmapp_admin or mcg_uxm_admin role add uxmapp_view_* capabilities if needed, this allows users in role to see menu for configuring uxm and inspecting/opening searches in search on tables/charts.

    For uxmapp_user or mcg_uxm_user ensure the following capabilities are enabled:

    • change_own_password
    • edit_own_objects
    • export_results_is_visible
    • list_all_objects
    • pattern_detect
    • rest_properties_get
    • run_collect
    • run_mcollect
    • schedule_search
    • search
    • uxmapp_view_* capabilities if needed

3. Clear CSS/JS caches

Splunk caches CSS/JS files for a long period, open /en-GB/_bump and press Bump version to get Splunk to refresh it's CSS/JS files cache.

  1. Enable and delete old input scripts

    Open Settings -> Data Inputs -> Scripts

    • Delete create_new_web_agents.py (if exists)
    • Delete task_tmp_fill_user_and_boottime.py (if exists, moved to update_kvstore.py)
    • Delete task_mq_consumer_pcagent_mt.py (if exists, being replaced by Go data parser)

    Enable input scripts

    • enable delete_old_files_and_packages.py (Deletes old unused app files and python dependencies, script will disable when done)

    • when upgrading to 2023.12.13: enable distributed_searchhead_23q4_migration.py (Migrates uxmap KVStore collections with new required entries, script will disable when done)

    • when upgrading to 2024.09.04 or 2025.01.28: enable distributed_searchhead_24q1_migration.py (Assigns desktop profiles to SLA's and removes application processes with setup/tmp names, script will disable when done)

      You can check if any desktop profiles are left with missing SLA's assigned, they can be mapped by editing and saving them under Administration -> UXM Desktop Agent -> Desktop app monitoring profiles:

      | inputlookup ux_profiles_lookup 
      | lookup ux_applications_lookup _key AS application_key OUTPUT name AS application_name 
      | table application_name, name, sla_key

    • enable check_license.py

    • enable update_alert_event_summaries.py

    • enable update_applications.py

    • enable update_endpoint_groups.py

  2. Update setup settings

    Goto Apps -> Manage Apps and select "Set up" for uxmapp or mcg_uxm app to set up hostnames and encrypt passwords.

    Enter HTTP Event collector hostname and token if empty.

    Apply license if needed and press Save. (2024.09.04 and newer requires license to save configuration and process data in consumer scripts)

  3. Update monitor settings

    Open uxmapp and select Administration -> UXM Desktop agent -> Monitors

    when upgrading to 2023.12.13: Edit Endpoint monitor and enable new event fields for newer agents (keep old monitor events for older agents) and press save.

    version 2025.01.28 adds the devices settings to endpoint monitor to collect device name, driver version, status every 24 hours.

  4. Overridden dashboards and settings

    UXM distributes it's dashboards and settings through uxmapp\default, all changes performed to dashboards and configs inside Splunk is stored under uxmapp\local.

    Please ensure that there are no local dashboards overrides our default provided dashboard under C:\Program Files\etc\apps\uxmapp\local\data\ui\views or C:\Program Files\etc\apps\mcg_uxm\local\data\ui\views.

    Please compare with the dashboards under C:\Program Files\etc\apps\uxmapp\default\data\ui\views to see if the changes are required and should be kept, restart the splunk server if any changes is made to the files.

  5. Restart the Splunk Search Head or hot reload new config

Upgrade Heavy Forwarder​

  1. Upload new Heavy Forwarder app

    Upload the mcg_uxm_distributed_hf_onpremise_YYYY.MM.DD.tar.gz under Apps -> Manage Apps, postpone restarting Splunk.

  2. Clear Splunk JS/CSS caches

    Open /en-US/_bump and click "Bump version" to clear JS/CSS caches.

  3. Run setup in app to set hostnames and encrypt passwords

    Open Manage Apps -> UXM -> Set up and enter the missing fields for KVStore hostname, HEC hostname and RabbitMQ hostname.

    Ensure RabbitMQ user is new uxmapp or old mcg_uxm if RabbitMQ isn't upgraded.

    Press Save in bottom, passwords will be encrypted using Splunk's key and UXM will redirect you to the Enterprise Dashboard.

  4. Start Splunk processing scripts

    Open Splunk and goto Settings -> Data Inputs -> Scripts and ensure that following scripts is enabled.

    • enable delete_old_files_and_packages.py (Deletes old unused app files and python dependencies, script will disable when done)
    • disable/enable task_mq_consumer_pcagent.py (Will be named consumer1 to consumer4, depending on how much data collector receives)
    • disable/enable task_mq_consumer_web.py (Will be named consumer1 to consumer4,
    • depending on how much data collector receives)
    • enable update_warranty.py if warranty lookups is configured and used.

  5. Restart Splunk

    Restart "Splunkd Service"

Validate setup​

  1. Test that endpoints works

    Open https://localhost/data/browser in a browser and validate that the HF works and returns "no get/post data received"

  2. Check for errors

    Open the UXM app on the Heavy Forwarder, the default dashboard will show status on installation and report any errors detected.

    PCAgent and Web consumer will show following info if everything works: