PSHMP Core: A Hybrid L4 Overlay for Proactive Self-Healing and Resilient Multi-Hop Delivery

Introduction Distributed systems rarely enjoy perfect connectivity. Packet loss, jitter, overloaded links, node failures, and partial outages appear in cloud, edge, industrial, and telecommunications environments alike. Classic recovery usually waits for a hard failure and then reacts. By the time a path is declared dead, applications have already felt the impact. PSHMP Core takes a different stance. It treats progressive degradation as a first-class signal. Nodes continuously evaluate their own and their neighbours' stability using a composite metric (K-Factor). When quality falls below configurable thresholds, the system starts reconstructing the affected delivery path—often before the path is completely unusable. The result is an overlay that tries to keep traffic moving with less disruption and with less dependence on any single central controller. The overlay sits above existing IP. It does not replace routers or rewrite the Internet Protocol. It adds a logical layer that can select relays, build multi-hop chains, detect trouble early, and rebuild paths when necessary.

Scope and Positioning PSHMP Core is intended as a technology foundation, not a finished consumer product. Organisations that already operate distributed platforms can integrate it to gain an additional resilience layer. Typical target environments include:

edge and multi-site computing;
telecommunications and backhaul overlays;
industrial and infrastructure networks;
real-time or telemetry-heavy systems;
geographically spread services that must tolerate partial outages;
any architecture that needs multi-hop delivery with faster recovery than pure reactive failover.

The protocol does not claim to eliminate every failure. When the underlying network is completely partitioned or when too few healthy relays remain, no overlay can invent connectivity. What it can do is use the available relays more intelligently and recover more quickly from the failures that do occur.

Architectural Position PSHMP Core lives between the application and the ordinary IP network:

The IP layer continues to provide basic reachability. PSHMP Core uses that reachability as a substrate for logical multi-hop paths that it can rebuild on its own terms.

Architecture Layers and Components The experimental implementation organises functionality into logical layers. The following table summarises the main components:

Control Plane decisions (path construction, healing priorities) influence Data Plane behaviour. Decentralization components supply candidates and state when a central coordinator is unavailable. Security and transport mechanisms protect and carry the traffic without changing the higher-level recovery logic.

Hybrid Operating Model One of the distinctive traits of the current design is the hybrid operating model. In normal conditions the system prefers linear relay chains. A chain is a simple ordered sequence of relays that carries data from source toward destination. Chains are easy to reason about, have predictable latency characteristics, and can be constructed with the help of an optional coordinator when stronger consistency is desired. When several nodes on an active chain show clear degradation (typically three or more), the system can switch into a local mesh-recovery mode for that chain. It does not tear down the entire network. It collects alternative candidates (via DHT and gossip), applies progressive fallback rules, checks a quality gate, and replaces only the affected path. Once conditions improve, operation can return to the linear style. This hybrid approach keeps the common case simple while still providing a robust escape hatch when multiple relays fail at once.

Dynamic Chain Relay A relay chain is a sequence of participating nodes that forward data. Chains are rebuilt when:

a relay becomes unreachable;
packet loss, latency, or jitter rises beyond useful thresholds;
node quality (K-Factor) falls;
a local network segment becomes unstable;
several nodes on the same path degrade together.

The goal is not merely to add hops. The goal is to keep a usable path alive by swapping out weak participants for healthier ones, without asking the underlying IP routing to change.

Node Quality Metric (K-Factor) PSHMP Core evaluates nodes with a composite metric called K-Factor. Rather than looking at a single instantaneous measurement, K-Factor combines several signals into one stability score that is used both for selecting relays and for detecting degradation early.

Reference Formula In the experimental implementation the reference formula is:

where:

loss is packet loss ratio in the range 0.0 .. 1.0 (clamped);
jitter is measured in milliseconds (clamped at the normalisation constant);
rtt is round-trip time in milliseconds (clamped);
battery is an optional node-health indicator normalised to 0.0 .. 1.0 (for stationary nodes this term may be fixed at 1.0).

Each term is clamped to the interval [0, 1] before weighting so that extreme values cannot dominate the score. The resulting K lies in approximately [0, 1]. The weights (0.45 / 0.25 / 0.20 / 0.10) and the normalisation constants (0.35, 800, 1200) are the reference values used in the current codebase. Integrators may change them to match their traffic patterns and topology; the architectural requirement is only that a continuous stability estimate drives selection and recovery decisions.

Status Bands The experimental implementation maps K to qualitative status bands:

= 0.85 EXCELLENT – preferred for new chains 0.70 <= K < 0.85 GOOD – usable 0.55 <= K < 0.70 ACCEPTABLE – increased monitoring 0.40 <= K < 0.55 DEGRADED – prepare replacement K < 0.40 CRITICAL – exclude from new paths no recent data UNKNOWN – do not use ]]> These thresholds are configuration parameters, not protocol constants. A node whose K is still above the hard-failure line but is sliding downward can be scheduled for replacement before it becomes a hard failure.

Path / Candidate Scoring When ranking candidate relays for a new or replacement chain the implementation uses a weighted score of the form:

where each component is normalised to [0, 1]. KScore reflects closeness of the node’s K-Factor to the desired operating region; LatencyScore and LossScore penalise high delay and loss; GeoScore (or more generally a failure-domain diversity bonus) favours candidates that improve topological spread. Exact normalisation functions are implementation-defined.

Proactive Self-Healing Self-healing is organised around priorities. A complete node failure is treated as critical and handled immediately. Progressive degradation is classified according to how far the K-Factor has fallen:

Each class carries its own scheduling delay and an adaptive cooldown so that the system does not thrash when many nodes fluctuate near a threshold. The typical sequence is:

degradation is observed (K-Factor or reachability);
the condition is evaluated and prioritised;
alternative candidates are gathered from DHT and gossip;
a replacement path is constructed, diversity-checked, and validated;
traffic is moved onto the new path;
the old path is retired.

Because the process starts from degradation rather than from total failure, the window of visible service impact can be shortened. Prototype measurements under simulated conditions have shown recovery behaviour on the order of hundreds of milliseconds; real-world figures depend on topology, load, and configuration.

Local Mesh Recovery When three or more nodes on the same chain are degraded, the system can activate local mesh recovery for that chain. The procedure roughly follows these steps:

mark the affected chain;
collect candidate relays from DHT and gossip, excluding recently failed or quarantined nodes;
apply a multi-level fallback ladder that gradually relaxes quality and diversity requirements;
run a quality gate before accepting the new path;
replace the old chain atomically.

Fallback Ladder The reference fallback levels used in the experimental implementation are:

= 0.70, diversity required Level 1: MinAvgK >= 0.65, diversity required Level 2: MinAvgK >= 0.60, diversity required Level 3: MinAvgK >= 0.52, diversity required Level 4: MinAvgK >= 0.42, diversity optional ]]> Early levels insist on high average K-Factor and good spread across failure domains. Later levels accept lower quality or drop the diversity constraint, still preferring any working path over silence. The quality gate that follows construction prevents a clearly worse path from being installed simply because it is the only remaining option.

Diversity-Aware Selection Failures are often correlated. Nodes that share a rack, a site, a provider, or a network segment tend to suffer together. Selecting the highest-scoring individual nodes without regard to shared risk can therefore produce a new path that fails in the same way as the old one. PSHMP Core can group nodes by logical failure domain (subnet, region, provider, availability zone, or any deployment-defined label). During recovery the selection logic prefers candidates from different groups. The preference is stronger when the network is already under stress and can be relaxed when healthy diversity is simply unavailable.

Failure Tracking and Quarantine A node that has just failed is a poor candidate for immediate re-use. PSHMP Core keeps a short history of failures and can place unstable nodes into a temporary quarantine. After a period of stable behaviour the node may be rehabilitated and allowed back into the candidate pool. This simple mechanism reduces the chance of repeated recovery loops caused by the same unreliable participants.

Decentralised Discovery and State Two complementary mechanisms keep the network informed. Gossip spreads node state, K-Factor updates, and chain proposals. It supports both push dissemination and anti-entropy pull synchronisation so that information continues to flow even when some links are lossy. Message types include node-state announcements, K-Factor updates, chain proposals, and explicit pull request/response pairs. A Kademlia-style DHT supplies candidate relays when a node needs fresh options, especially useful when a coordinator is unreachable. The DHT is treated as a discovery aid rather than a generic key-value store; the values that matter are node identity, reachability, and quality hints. Typical parameters in the experimental code are 160 buckets, bucket size k = 20, and concurrency parameter alpha = 3. An optional coordinator (which may itself run as a small Raft cluster) can provide stronger consistency for initial chain construction and global views. The architecture does not make the coordinator mandatory; local mesh recovery and DHT/gossip remain available when the coordinator is absent.

Data Delivery Path and Acknowledgement Model Reliable multi-hop delivery is handled by a dedicated DataPath component. Application data is segmented into chunks. Each chunk carries a sequence number. The sender transmits batches of chunks over the current relay chain. The receiver reports progress using batch acknowledgements that include an explicit gap list.

Delivery Flow (Conceptual)

Relay 1 -----> Relay 2 -----> ... -----> Destination | v Receiver: update highest contiguous sequence record missing ranges as Gaps | v Batch ACK = { session, last_contiguous, gaps[], timestamp, MAC } | v Sender: retransmit only the missing sequences (from Retransmit Cache) ]]>

Batch ACK Structure A Batch ACK carries:

session identifier;
highest contiguous sequence number received (LastReceived);
a list of missing ranges (Gaps), each described by a start and end sequence number;
timestamp and optional integrity protection (e.g., HMAC).

Conceptually the set of sequences that still need retransmission is:

LastReceived that were sent but not yet ACKed) ]]> In practice the sender uses the gap list together with its outstanding-send window to decide exactly which chunks to retransmit. A short linger window (on the order of several hundred milliseconds in the reference implementation) allows late packets to arrive before gaps are declared, reducing unnecessary retransmissions.

Control Traffic Reduction Under the traffic patterns examined in internal tests, batch acknowledgements with gap lists have reduced control-plane overhead substantially (in favourable cases approaching the order of 75 % compared with per-packet ACKs). Actual savings depend on loss rate, batch size, and message size. The architectural gain is that the control channel stays light even when data volume is high.

Transport Abstraction The overlay is deliberately transport-agnostic. Implementations may run over UDP, DTLS, TLS-protected TCP, WebRTC DataChannels, or other bidirectional transports. TURN assistance can be used where NAT traversal is required. The choice of transport is an engineering decision for each deployment; the path-selection and recovery logic stays the same.

Security Considerations A decentralised relay system must assume that some participants may be malicious, misconfigured, or simply overloaded. The current design includes rate limiting, optional proof-of-work admission, authenticated and encrypted transport options (TLS/DTLS), HMAC-protected control messages, failure tracking, and temporary quarantine. These mechanisms are necessary but not sufficient. Deployments still need proper node identity, key management, authorisation policy, and operational monitoring. PSHMP Core is a resilience layer, not a complete security architecture.

Scalability Notes The architecture is intended to scale from small groups of nodes to several thousand participants. Distributed state (gossip), DHT-based discovery, optional sharding, and the ability to operate without a permanent central controller are the main tools. Large-scale network emulation with more than five thousand nodes has been used to exercise the implementation; those figures are experimental observations, not protocol guarantees. Real performance depends on topology, churn, and configuration.

Experimental Implementation An experimental implementation, PSHMP Core v3.1, has been written in Go. It contains the principal components described in this document:

Chain Relay and PathFinder;
K-Factor evaluation and adaptive probing;
Self-Healing Engine with priority scheduling and mesh state machine;
local mesh recovery with multi-level fallback and quality gate;
diversity-aware candidate selection;
Gossip (push and anti-entropy) and Kademlia-style DHT;
DataPath with batch acknowledgements and gap recovery;
optional Raft-based coordinator and sharding;
transport support including UDP, DTLS, and WebRTC;
Prometheus metrics and structured logging.

The implementation has been exercised primarily through large-scale emulation and limited real-network experiments. It is positioned as a pilot-ready technology foundation rather than a finished mass-market product. Scoring weights, exact thresholds, and certain optimisations remain tunable by integrators.

Example Scenario Consider a six-relay chain:

Relays B, C and D begin to show rising loss and falling K-Factor. The Self-Healing Engine marks the degradation, raises the priority, and triggers local mesh recovery for this chain. Candidates are gathered from DHT and gossip; recently failed nodes are excluded; diversity constraints prefer relays from different failure domains. A new sequence is assembled, passed through the quality gate, and installed:

The underlying IP routes are untouched. Only the logical overlay path has changed. Traffic continues with a shorter disruption window than a pure reactive failover would typically allow.

Potential Benefits When integrated into a larger platform, PSHMP Core can offer:

shorter recovery windows through proactive reconstruction;
continued delivery when individual relays or segments fail;
reduced dependence on any single path or controller;
lower control-plane overhead via batch acknowledgements;
reuse of existing IP infrastructure;
adaptive behaviour that tightens or relaxes constraints according to the severity of the failure.

Integration Model The expected use is as an embedded resilience layer inside an existing distributed system. The integrating organisation keeps ownership of application logic, identity, infrastructure, and commercial packaging. PSHMP Core supplies the adaptive multi-hop delivery and recovery machinery underneath.

Deployment Considerations Before wide deployment an organisation should examine topology density, expected loss and latency, failure-domain layout, security and identity requirements, monitoring, and capacity. Controlled pilot deployments remain the practical way to validate behaviour under real traffic. The formulas and thresholds given in this document are a starting point; they should be calibrated against the target network.

Design Principles

Decentralisation — discovery and recovery must continue when a central component is unavailable.
Proactive recovery — degradation is a signal, not merely a post-mortem.
Dynamic paths — delivery routes adapt to changing conditions.
Diversity — correlated failures are treated as a first-class risk.
Layer independence — the overlay does not require changes to L3 routing.
Graceful degradation — when ideal candidates are scarce, constraints are relaxed progressively rather than failing abruptly.

Limitations PSHMP Core cannot create connectivity where none exists. Performance depends on the number and quality of available relays, the underlying network, workload, and configuration. Figures obtained from emulation or limited trials must be re-validated for each production environment. The reference formulas are part of the experimental model; production deployments may adjust weights and thresholds.

IANA Considerations This document has no IANA actions.

Terminology PSHMP Core: the overlay architecture described in this document. Relay: a participating node that forwards data for others. Relay chain: an ordered sequence of relays forming a logical delivery path. K-Factor: a composite stability metric used for selection and early degradation detection. Self-healing: automatic reconstruction of degraded delivery paths. Local mesh recovery: path reconstruction performed for an affected chain using alternative candidates, fallback levels, and a quality gate. Diversity: preference for relays drawn from different failure domains. Gossip: decentralised dissemination of node and path state. DHT: distributed mechanism used primarily for candidate discovery. Gap list: explicit list of missing sequence ranges reported in a batch acknowledgement. DataPath: component responsible for chunking, transmission over a relay chain, batch acknowledgements, and retransmission of missing sequences.

Conclusion PSHMP Core offers a practical hybrid overlay for environments where connectivity is imperfect and recovery time matters. Linear chains keep the common case simple; local mesh recovery, K-Factor-driven decisions, diversity awareness, and efficient acknowledgements provide the machinery needed when conditions worsen. The design deliberately stays above IP, remains transport-flexible, and can operate with or without a central coordinator. An experimental Go implementation (v3.1) demonstrates that the architecture is realisable and has been exercised at significant scale in emulation. Reference formulas for K-Factor, candidate scoring, the recovery fallback ladder, and the batch-acknowledgement delivery model are provided so that readers can understand the concrete model; integrators remain free to tune weights and thresholds to their own networks. PSHMP Core is offered as a technology foundation for organisations that need resilient multi-hop delivery and are prepared to evaluate, tune, and integrate it within their own platforms.