Intermittent VPN problems are notoriously difficult to diagnose during VPN troubleshooting because everything can look fine on the surface while something is quietly going wrong underneath. Mafiree encountered exactly this scenario with a client whose infrastructure was monitored through a jump server connected via a site-to-site IPsec VPN. The tunnel showed up, routing worked correctly, SSH access was available, and packet loss was negligible - yet the monitoring dashboards kept showing gaps and interruptions. The team was called in to find out why, initially assuming the fault lay outside the VPN itself. That assumption proved incorrect.
VPN Troubleshooting: Why Tunnel Status Can Be Misleading
The investigation's central lesson was that an "established" VPN connection can still hide serious problems. A tunnel can remain up while suffering from rekeying failures, fragmentation issues, mismatched lifetime settings, failed Phase 2 negotiations, or firmware bugs - all of which can cause brief disruptions that are too short to register as a full outage but long enough to break monitoring data collection.
Checking MTU and Encryption Overhead
Because the problem emerged after VPN-related changes, the team first suspected MTU issues. Testing showed that 1472-byte packets failed while 1400-byte packets went through, confirming that encapsulation overhead was cutting into the usable payload. Given the IPsec setup - AES-256 encryption, SHA-256 authentication, and IKEv2 with Perfect Forward Secrecy - the effective payload size was roughly 1365 bytes. This was a real factor, but it didn't account for the intermittent nature of the failures, since smaller packets and control traffic kept working normally.
Network Path Analysis Using MTR
To check whether unstable routing was involved, engineers ran an extensive 100-packet MTR test. The results were clean across the board: zero packet loss, stable latency with no jitter, consistent round-trip times, and no route changes. The network path itself was healthy, so attention shifted elsewhere.
SSH Tests Looked Fine, At First
Since monitoring depended on the jump server, SSH performance was tested repeatedly. Ten separate sessions connected successfully in under a second each time, suggesting authentication and tunnel responsiveness were both working. But this created a contradiction: the tunnel tested as healthy, yet the actual service relying on it kept failing in production.
Finding the Hidden Pattern
Rather than continuing with occasional manual checks, the team built a continuous SSH testing script with detailed logging. This revealed something spot-checks had missed entirely: the failures weren't random. They occurred at recurring intervals, hinting at a connection to tunnel lifetime settings or Security Association renegotiation cycles - a strong clue that timing itself was significant.
Tracing It Back to a Firmware Change
Reviewing the change history showed the problem started right after the client upgraded their firewall firmware and adjusted VPN settings to match the new recommendations. This redirected the investigation toward the VPN configuration itself. Every parameter - Phase 1 and Phase 2 encryption, preshared keys, PFS settings, and lifetime values - was checked and found to match on both ends. Even after further adjustments, though, the intermittent issues continued, showing that matching configurations alone weren't the whole story.
The Real Cause: Silent Configuration Drift
The breakthrough came from examining firewall syslogs instead of the VPN status dashboard. The logs showed that VPN parameters were being automatically regenerated and reverted to firmware defaults during operation - even after administrators had manually corrected them - with no indication of this happening on the dashboard. This explained why the tunnel stayed "up," connectivity looked mostly normal, and yet monitoring kept breaking: the real issue was a firmware-driven configuration consistency problem invisible to standard status checks.
VPN Troubleshooting Resolution
As a short-term fix, Mafiree aligned tunnel settings with the values the firmware kept reverting to, which improved but didn't fully resolve things. The client eventually replaced the firewall entirely, after which monitoring stabilized immediately, VPN interruptions stopped, and no further anomalies appeared.
Key Takeaways
The case reinforced several lessons: an "UP" status doesn't guarantee health; recent changes should always be checked first; continuous testing catches patterns that spot checks miss; logs often reveal what dashboards hide; and rigorous, evidence-based troubleshooting rather than assumptions - is what ultimately uncovers root causes.
