CSMS Architecture at Scale: WebSockets, State and Idempotency

OCPP inverts the usual web architecture. Instead of clients making short requests to stateless servers, thousands of chargers hold long-lived WebSocket connections, and the server must be able to send commands down any one of them at any time. Most CSMS scaling problems come from treating it like a normal REST backend. This post lays out an architecture that holds up as a fleet grows, and the decisions that matter most.

The shape of the traffic

• Persistent connections: one WebSocket per charging station, open for days or weeks.
• Bidirectional RPC: either side can send a CALL; each expects a CALLRESULT or CALLERROR with the same message ID.
• One outstanding CALL per direction: OCPP-J expects a sender to wait for a response before sending its next CALL, so a slow handler on your side stalls that charger.
• Spiky reconnects: after a deployment, a network outage or a cloud incident, a large share of the fleet reconnects and sends BootNotification within minutes.
• Bursty session traffic following daily usage patterns, with meter values flowing during sessions.

Layered architecture

The key separation is between the gateway (which knows about sockets and frames) and the domain services (which know about sessions and tariffs). Keep business logic out of the gateway. That lets you scale and redeploy them independently, and lets you support OCPP 1.6 and 2.0.1 with different gateway adapters feeding a shared domain model.

Routing commands to the right connection

When an operator clicks remote start, the request lands on some API server, but the charger's socket lives on one specific gateway node. You need a connection registry: a fast store mapping station ID to gateway node, updated on connect and disconnect. The API publishes the command to that node's channel; the gateway sends the CALL, waits for the response, and publishes the result back with a correlation ID.

• Make registry entries expire unless refreshed, so a crashed node does not leave stale routes.
• When a station reconnects to a different node, the new connection wins; the old node must close its stale socket.
• Give every outbound command a timeout and a clear result: accepted, rejected, timed out, or station offline. The UI and the OCPI Commands module both need it.

Idempotency and retries

Chargers retry. If a StartTransaction response is lost, the charger sends it again, sometimes with a new message ID after reconnecting. If your handler blindly inserts a row, you get two sessions and a billing dispute. Make handlers idempotent on business keys, not message IDs:

• For 1.6 StartTransaction, treat station, connector, idTag, meterStart and timestamp together as the natural key, and return the same transactionId for a duplicate.
• For 2.0.1 TransactionEvent, the station supplies transactionId and a per-transaction seqNo; deduplicate on those and process out-of-order events correctly. See the TransactionEvent post.
• For StopTransaction and MeterValues, accept duplicates and late arrivals without corrupting totals.

Respond quickly. Do the minimum needed to build a correct response synchronously (look up the token, allocate the transaction ID) and push everything else (OCPI updates, notifications, analytics) onto queues.

Handling reconnect storms

• Make BootNotification handling cheap and cache-friendly. Avoid heavy database writes per boot.
• Use the Pending boot status or interval field to spread follow-up work, rather than pushing configuration to every station at once.
• Drain gateway nodes gradually during deployments instead of killing all connections together.
• Load test this explicitly with simulators; see testing with a charge point simulator.

Station state

Keep a current-state record per station and connector (connection status, last seen, connector status, active transaction, firmware version, configuration hash) separate from the event history. Update current state from events, but treat the station as the source of truth for its own status: after reconnect, use TriggerMessage for StatusNotification rather than trusting what you had cached. Time matters too: order by the station's timestamps where provided, and record server receive time separately, because station clocks drift.

Multi-tenancy

Most platforms serve multiple operators or sub-networks. Decide early where the tenant boundary is:

• Identification: the station identity is in the WebSocket URL path, conventionally the last path segment. Resolve station to tenant at connection time and carry the tenant ID on every internal message.
• Credentials: per-station credentials (Basic auth password or client certificate), never shared across tenants.
• Data isolation: tenant ID on every row and in every query, enforced by the data layer rather than remembered by each developer.
• Noisy neighbours: rate limits and queue partitioning per tenant so one operator's reconnect storm does not slow others.

Observability

• Raw message log per station: every frame in and out, with timestamps, retained long enough to investigate disputes. This is the single most valuable debugging tool for OCPP.
• Metrics: connected stations, connects and disconnects per minute, CALL latency by action, CALLERROR rate by code, outbound command timeouts, queue depth.
• Tracing: carry a correlation ID from operator action through the bus to the CALL and back.
• Domain metrics: session success rate, stuck transactions, unpriced sessions, CDR delivery lag to roaming partners.

Security

Use TLS for every connection and per-station credentials at minimum (Security Profile 2), and support client certificates (Profile 3) where hardware allows. Validate the station identity in the URL against the credential presented. If TLS terminates at the load balancer, Profile 3 requires either pass-through or forwarding of the client certificate. See OCPP 1.6 security profiles and OCPP 2.0.1 security.

Key takeaways

• Separate the WebSocket gateway from domain services, and use a connection registry to route commands.
• Make handlers idempotent on business keys; chargers will retry and replay.
• Design for reconnect storms, not average load.
• Log every frame per station; it settles most disputes and debugging sessions.

HashTech designs and operates CSMS platforms on this kind of architecture, including the platform behind REDE Charging; see our EV charging work.