FCP_FCT_AD-7.2 Diagnostics

Diagnostics Detailed Explanation

Diagnostics is a critical aspect of maintaining FortiClient EMS and endpoints. It helps identify and resolve issues effectively, ensuring endpoints remain compliant, secure, and operational.

1. Common Issues and Troubleshooting

This section focuses on frequent issues encountered in EMS and endpoints, their symptoms, and step-by-step troubleshooting methods.

1.1 EMS-to-Endpoint Communication Failures

Symptoms

1. Endpoints appear offline in the EMS dashboard.
2. Policies are not being applied or updated on endpoints.

Troubleshooting Steps

1. Verify EMS Network Connectivity:

   • Ensure the EMS server is accessible from the endpoints.
   • Use the ping command from an endpoint to test connectivity to the EMS server's IP address.

2. Check Required Ports:

   • EMS uses specific ports for communication:
     • HTTPS (443): For secure communication.
     • Additional ports (if customized) should also be verified.
   • Confirm that the firewall or network policies are not blocking these ports.

3. Inspect Endpoint Firewall Settings:

   • On the endpoint device, check if the local firewall is blocking traffic to the EMS server.
   • Add exceptions for EMS-related traffic if needed.

4. Test with a Browser:

   • From an endpoint, try accessing the EMS server interface using a browser:
     • URL format: https://<EMS_Server_IP>:<Port>.
   • Ensure there are no SSL/TLS certificate warnings.

1.2 Policy Synchronization Issues

Symptoms

1. Endpoints do not receive newly created or updated policies.
2. Endpoint configurations appear outdated despite recent changes.

Troubleshooting Steps

1. Validate Endpoint Connectivity:

   • Check if the endpoint is online and actively connected to the EMS server.
   • Navigate to the endpoint list in EMS to confirm the last communication time.

2. Manually Push Policies from EMS:

   • In the EMS dashboard, select the endpoint and choose Push Policy.
   • Monitor the status to ensure the policy is applied successfully.

3. Check EMS Logs for Errors:

   • Access EMS logs and filter entries related to policy synchronization.
   • Look for errors such as timeouts, misconfigurations, or authentication failures.

1.3 Licensing Problems

Symptoms

1. Error messages indicating the number of licensed devices has been exceeded.
2. New devices are unable to register with EMS.

Troubleshooting Steps

1. Verify the Number of Licensed Devices:

   • Navigate to the License Management section in EMS.
   • Check the number of used licenses against the total available licenses.

2. Unregister Unused Devices:

   • Identify inactive or obsolete devices in the EMS dashboard.
   • Remove or unregister these devices to free up license slots.

3. Contact Fortinet Support (if necessary):

   • If license allocation issues persist, contact Fortinet for assistance in reallocating licenses.

2. Diagnostic Tools

FortiClient EMS provides several built-in tools to diagnose and resolve issues effectively.

2.1 EMS Logs

Accessing Logs

1. Open the EMS web interface and navigate to the Logs section.
2. View logs categorized into:
   • System Logs:
     • Record administrative changes and critical system events.
   • Endpoint Logs:
     • Track endpoint communication, policy updates, and compliance status.

Using Filters

1. Apply filters to quickly locate relevant log entries:
   • Time: Narrow down logs to a specific time range.
   • Device Name or IP: Focus on logs for a particular endpoint.
   • Error Type: Search for specific error codes or warnings.

Export Logs

1. Export logs in CSV format for deeper analysis or sharing with Fortinet support.
2. Use tools like Excel or specialized log analyzers to process large datasets.

2.2 Endpoint Debugging

Enabling Debug Mode

1. On the FortiClient application installed on the endpoint:
   • Go to the Settings or Support section.
   • Enable Debug Mode to start capturing detailed logs.
2. Reproduce the issue to ensure relevant log entries are generated.

Downloading Debug Logs

1. Once debugging is complete, save the log file to the local device.
2. Upload the log to EMS for analysis or share it directly with Fortinet support for assistance.

2.3 Fabric Diagnostics

Using FortiGate Diagnostic Tools

1. Verify API Connectivity:

   • Check the API key configuration between EMS and FortiGate.
   • Ensure the key is valid and matches the EMS server's IP restrictions.

2. Recognize EMS as a Fabric Device:

   • Open the Fabric Devices section in FortiGate.
   • Confirm that EMS appears as a recognized and active component.

3. Network Testing Tools:

   • Use FortiGate's built-in diagnostic tools (e.g., ping, traceroute) to troubleshoot connectivity issues between FortiGate and EMS.

3. Advanced Diagnostic Scenarios

Advanced diagnostic scenarios involve less common but more complex issues. These often require deeper analysis and understanding of FortiClient EMS behavior.

3.1 Endpoint Performance Issues

Symptoms

1. Endpoints experience high CPU or memory usage.
2. Slow response times or freezing when using FortiClient features.

Troubleshooting Steps

1. Check FortiClient Version Compatibility:

   • Ensure the installed FortiClient version is fully compatible with the EMS server version.
   • Update FortiClient if necessary.

2. Verify Endpoint Hardware:

   • Confirm the endpoint meets the minimum system requirements for FortiClient:
     • Recommended: 8 GB RAM, multi-core CPU.
   • Upgrade hardware if usage consistently exceeds available resources.

3. Update FortiClient to the Latest Version:

   • Download the latest FortiClient version from EMS or the Fortinet website.
   • Install updates to fix known bugs and improve performance.

3.2 Integration Failures

Symptoms

1. EMS does not appear in the Security Fabric.
2. FortiGate fails to communicate with EMS.

Troubleshooting Steps

1. Recheck API Key or Certificate Configuration:

   • Verify that the API key in EMS matches the configuration on FortiGate.
   • Check for IP restrictions or expired certificates.

2. Verify TLS Settings:

   • Ensure the TLS certificate on EMS is valid and trusted by FortiGate.
   • Reissue or update certificates if necessary.

4. Reporting and Prevention

4.1 Automated Reports

Automated reports are critical for tracking the health of endpoints, monitoring security events, and ensuring policy compliance.

Purpose of Automated Reports

• Provide a comprehensive overview of endpoint activity and security posture.
• Help administrators identify trends, anomalies, or recurring issues.
• Facilitate compliance with regulatory or organizational reporting requirements.

Steps to Configure Automated Reports

1. Access the Reporting Section:

   • Log in to the EMS console and navigate to Reports → Create New Report.

2. Select a Report Type:

   • Endpoint Health Report:
     • Summarizes the status of all endpoints, including compliance and threat detections.
   • Threat Activity Report:
     • Lists detected malware, vulnerabilities, and blocked applications.
   • Policy Compliance Report:
     • Highlights devices that violate assigned policies.

3. Customize Report Parameters:

   • Define the time range (e.g., daily, weekly, monthly).
   • Filter by device group, location, or compliance status.

4. Schedule Reports:

   • Set the frequency of report generation (e.g., every Monday at 9:00 AM).
   • Choose delivery methods (e.g., email to administrators or export to a shared drive).

5. Export Reports:

   • Save reports in formats like PDF, CSV, or HTML for sharing or archival purposes.

Benefits

• Regular visibility into system performance and security.
• Quick identification of underperforming endpoints or recurring policy violations.

4.2 Proactive Monitoring

Proactive monitoring involves setting up alerts and systems to detect potential issues before they escalate.

Setting Alerts for Key Events

1. Endpoint Offline Status:

   • Configure alerts to notify administrators when an endpoint has been offline for an extended period.
   • Helps identify connectivity issues or inactive devices.

2. Unusual Activity Spikes:

   • Set thresholds for event frequency (e.g., multiple malware detections within a short time frame).
   • Trigger alerts for further investigation.

3. License Utilization:

   • Monitor the number of active licenses.
   • Set alerts when usage approaches the maximum limit.

How to Enable Alerts

1. Navigate to the Alerts section in EMS.
2. Create a new alert profile:
   • Define the triggering conditions (e.g., policy violations, threat detections).
   • Specify notification methods (e.g., email, SMS).
3. Assign the alert profile to relevant administrators or teams.

4.3 Regular Updates

Keeping EMS and FortiClient updated is a fundamental step in maintaining a secure and efficient environment.

Why Updates Are Critical

1. Bug Fixes:
   • Address known issues that could affect performance or functionality.
2. Security Patches:
   • Close vulnerabilities that could be exploited by attackers.
3. New Features:
   • Take advantage of improvements in usability, integration, and security.

Updating FortiClient EMS

1. Check for Updates:
   • Log in to the EMS console.
   • Navigate to System Maintenance → Check for Updates.
2. Review Release Notes:
   • Examine the update’s release notes to understand what changes are included.
   • Ensure compatibility with existing FortiClient versions.
3. Apply the Update:
   • Back up the EMS configuration before proceeding.
   • Download and install the update directly from the EMS interface.
4. Test After Updating:
   • Verify that endpoints are communicating with EMS as expected.
   • Check for any issues in logs or endpoint behavior.

Updating FortiClient Endpoints

1. Push Updates from EMS:
   • Use the EMS console to distribute the latest FortiClient version to managed endpoints.
   • Navigate to Devices → Update FortiClient.
2. Configure Automatic Updates:
   • Enable auto-updates in the FortiClient settings to ensure endpoints remain current.

Preventive Maintenance Tips

1. Regular Backups:

   • Schedule backups of the EMS configuration to protect against data loss.
   • Store backups securely in an offsite location or cloud service.

2. Routine Log Reviews:

   • Periodically review EMS logs for unusual patterns or anomalies.
   • Pay attention to repeated errors, unauthorized access attempts, or policy violations.

3. Endpoint Testing:

   • Periodically test endpoints to ensure they are functioning as expected.
   • Validate antivirus updates, policy adherence, and compliance checks.

4. Audit Security Policies:

   • Review and update security policies regularly to address emerging threats or organizational changes.

Diagnostics (Additional Content)

Ensuring FortiClient EMS and endpoints operate correctly requires understanding common issues, log analysis, and diagnostic tools. This guide will provide detailed explanations and step-by-step troubleshooting techniques to help you efficiently diagnose and resolve problems.

4.1 Common Issues and Solutions

FortiClient EMS, FortiClient agents, and FortiGate integration may encounter connectivity, policy synchronization, and endpoint detection issues. Below, we will discuss the most common problems, their causes, and solutions.

Issue 1: Endpoint Cannot Connect to EMS

Symptoms:

• FortiClient fails to register with EMS.
• The EMS server appears offline in FortiClient.
• You see “Connection failed” errors in logs.

Possible Causes and Solutions

Cause	Solution
Firewall blocking EMS ports	Open ports 8013 (EMS Web) and 10443 (EMS-Agent communication) on the firewall.
Incorrect EMS IP/Hostname	Ensure FortiClient is using the correct EMS IP or FQDN.
Network connectivity issues	Run ping EMS_IP or telnet EMS_IP 10443 to check if EMS is reachable.
EMS service is not running	Restart FortiClient EMS service on the EMS server.

Step-by-Step Fix

1. Check if EMS service is running

• On the EMS server, open Services.msc.
• Locate FortiClient EMS Service.
• If it’s stopped, right-click > Start.

2. Verify EMS server connectivity

• Open Command Prompt on the endpoint and run:

  ping EMS_IP
  telnet EMS_IP 10443
  

• If the connection fails, check firewall rules.

3. Check FortiClient Logs

• Go to C:\ProgramData\Fortinet\FortiClient\logs and open ems.log
• Look for connection failure messages.

Issue 2: Endpoint Does Not Receive Policies from EMS

Symptoms:

• Endpoint appears connected to EMS but does not receive assigned policies.
• Security settings, web filtering, or VPN configurations do not apply.

Possible Causes and Solutions

Cause	Solution
Policy synchronization issue	Manually refresh policies in EMS.
Incorrect policy assignment	Ensure the endpoint is in the correct group in EMS.
Outdated FortiClient version	Update to the latest FortiClient version.

Step-by-Step Fix

1. Manually Refresh Policies

• On the endpoint, open Command Prompt and run:

  FortiClientConsole.exe /policyUpdate
  

• This forces FortiClient to sync policies with EMS.

2. Check Policy Assignment in EMS

• Open FortiClient EMS Console.
• Navigate to Endpoints > Select Device.
• Verify the applied policy.

3. Reapply Security Policies

• In EMS, go to Endpoint Profiles.
• Select the affected policy and click "Reapply".

Issue 3: FortiGate Does Not Detect Endpoints

Symptoms:

• FortiGate does not display FortiClient endpoints in Security Fabric > FortiClient Monitor.
• Non-compliant devices are not blocked.

Possible Causes and Solutions

Cause	Solution
FortiClient not configured for Security Fabric	Enable Endpoint Control on FortiGate.
Incorrect EMS-FortiGate integration	Verify EMS IP settings in FortiGate Security Fabric.
Firewall blocking FortiClient-FortiGate communication	Open port 8013 and 10443.

Step-by-Step Fix

1. Enable Endpoint Control on FortiGate

• Log in to FortiGate Web UI.
• Go to Security Fabric > Endpoint Control.
• Enable FortiClient Registration and Compliance Enforcement.
• Click Apply.

2. Verify EMS Connection

• Go to Security Fabric > EMS Servers.
• Ensure EMS is connected to FortiGate.
• If not, re-enter the EMS IP and authentication details.

3. Check Registered Endpoints

• Run the following FortiGate CLI command:

  diagnose endpoint list
  

• If no endpoints appear, check EMS-FortiGate connectivity.

4.2 Log Analysis

Logs are critical for diagnosing issues. FortiClient EMS and endpoints generate logs that provide insights into connection issues, policy failures, and security alerts.

EMS Logs (Server-Side)

• Location (Windows EMS Server):

  C:\Program Files (x86)\Fortinet\FortiClientEMS\logs
  

• Key Logs:

  • ems.log – Connection status, errors, EMS service logs.
  • policy.log – Policy assignments and failures.
  • database.log – EMS database issues.

FortiClient Endpoint Logs

• Windows:

  C:\ProgramData\Fortinet\FortiClient\logs
  

• Linux/macOS:

  /var/log/forticlient/
  

• Key Logs:

  • fmon.log – Connection between FortiClient and EMS.
  • vuln.log – Vulnerability scan results.
  • vpn.log – VPN connection issues.

Example: Checking Logs for Connection Issues

1. Open Command Prompt.

2. Navigate to FortiClient logs folder:

cd C:\ProgramData\Fortinet\FortiClient\logs

3. Open ems.log:

notepad ems.log

4. Look for errors such as "Connection Failed" or "Policy Not Applied".

4.3 Diagnostic Tools

1. FortiClient Diagnostic Tool

• A built-in tool that captures real-time logs and troubleshooting data.
• To use:
  1. Open FortiClient.
  2. Go to Settings > Diagnostics.
  3. Click "Generate Diagnostic Report".

2. EMS Debug Mode

• Enables in-depth troubleshooting of FortiClient EMS.
• To enable:
  1. Open EMS Console.
  2. Navigate to System Settings > Debug Mode.
  3. Enable Debug Logging.
  4. Restart the EMS Service.

3. Useful FortiGate CLI Commands

Command	Purpose
diagnose endpoint list	List registered FortiClient endpoints
diagnose test application fgfmd 3	Check EMS-FortiGate communication
`execute log display	grep FortiClient`

4.4 Advanced Troubleshooting Scenarios

Scenario 1: Endpoint Cannot Establish a VPN Connection via FortiClient

Symptoms:

• Users cannot connect to VPN using FortiClient.
• VPN connection status stuck on “Connecting”.
• Users receive authentication failure errors.

Possible Causes and Solutions

Cause	Solution
Incorrect VPN settings in EMS	Ensure correct VPN profile is applied to the endpoint.
FortiGate blocking VPN traffic	Verify that FortiGate allows VPN traffic on ports 443 (SSL VPN) or 500/4500 (IPSec VPN).
Authentication failure	Ensure correct user credentials and check authentication method (LDAP, RADIUS, or local users).
FortiClient logs show "Certificate Error"	Import the correct SSL certificate from FortiGate into FortiClient.

Step-by-Step Fix

1. Check VPN Configuration in EMS

• Open EMS Console > Endpoint Profiles > VPN Settings.
• Ensure the correct VPN Gateway (FortiGate IP) and Port (443 or 8443 for SSL VPN) are set.
• Save and reapply policies to endpoints.

2. Verify FortiGate VPN Policy

• Log in to FortiGate Web UI.
• Go to VPN > SSL-VPN Settings.
• Ensure "Listen on Port" is set to 443 and "Tunnel Mode" is enabled.

3. Check FortiClient VPN Logs

• Navigate to:

  C:\ProgramData\Fortinet\FortiClient\logs
  

• Open vpn.log and look for error messages such as "Failed to establish connection".

4. Run a FortiGate Debug Command

• In FortiGate CLI, run:

  diagnose debug application sslvpn -1
  diagnose debug enable
  

• Attempt to connect via FortiClient and check for VPN failure reasons.

Scenario 2: FortiClient Endpoint Appears as Non-Compliant in Security Fabric

Symptoms:

• The endpoint is registered with FortiGate but flagged as non-compliant.

• Network access is restricted despite the endpoint having the correct policies.

• FortiGate CLI shows non-compliant status when running:

  diagnose endpoint list
  

Possible Causes and Solutions

Cause	Solution
Security posture check failure	Ensure all required security policies are enabled in EMS.
Endpoint is missing required software updates	Check if "Patch Management" is enforced in EMS.
Outdated FortiClient version	Upgrade to the latest FortiClient version.

Step-by-Step Fix

1. Check Endpoint Compliance Status in EMS

• Navigate to EMS Console > Endpoints.
• Select the affected device and check the Compliance Score.
• If below 100%, check the missing security controls.

2. Force a Policy Update on the Endpoint

• On the endpoint, run:

  FortiClientConsole.exe /policyUpdate
  

• Restart FortiClient and check if compliance is restored.

3. Verify Compliance Policy on FortiGate

• Run in FortiGate CLI:

  diagnose endpoint compliance list
  

• If the endpoint is still flagged, adjust compliance policies in EMS.

Scenario 3: FortiAnalyzer Not Receiving FortiClient EMS Logs

Symptoms:

• No logs from EMS appear in FortiAnalyzer.
• Running execute log display on FortiAnalyzer shows no entries related to FortiClient.

Possible Causes and Solutions

Cause	Solution
EMS log forwarding is disabled	Enable Log Forwarding in EMS.
Incorrect FortiAnalyzer IP in EMS settings	Verify and correct the FortiAnalyzer IP address.
Port 514 (Syslog) is blocked	Ensure UDP port 514 is open between EMS and FortiAnalyzer.

Step-by-Step Fix

1. Enable Log Forwarding in EMS

• Open EMS Console > Logging & Reporting.
• Enable "Forward logs to FortiAnalyzer".
• Enter FortiAnalyzer IP address and click Save.

2. Check if EMS is Sending Logs

• Run in FortiClient EMS CLI:

  get system log status
  

• If logs are not being sent, restart the EMS Logging Service.

3. Verify Logs in FortiAnalyzer

• Log in to FortiAnalyzer Web UI.
• Go to Log View > Device Logs.
• If EMS logs are missing, re-add EMS as a log source.

4. Use CLI to Check Log Flow

• On FortiAnalyzer, run:

  execute log display | grep "FortiClient"
  

• This will show all logs related to FortiClient EMS.

4.5 Useful FortiClient EMS and FortiGate Debug Commands

1. Debugging FortiClient Connectivity Issues

Run this on FortiClient EMS to check if it’s communicating with endpoints:

diagnose test application fctemsd 3

• If the output shows "Failed to connect to endpoint", check firewall rules and EMS service status.

2. Debugging FortiGate-FortiClient Communication

Run this on FortiGate CLI to list connected endpoints:

diagnose endpoint list

• If an endpoint does not appear, verify EMS registration settings.

3. Checking EMS Policy Enforcement

Run this on FortiClient Endpoint:

FortiClientConsole.exe /policyUpdate

• This forces FortiClient to sync policies with EMS.

4. Checking Security Fabric Connection

Run this on FortiGate CLI to check EMS connectivity:

diagnose test application fgfmd 3

• If EMS is unreachable, verify EMS IP configuration in FortiGate.