The Mist WebSocket API provides real-time streaming of device and client statistics. To get reliable, production-grade delivery, your client script must be built to handle the nature of a persistent, high-throughput streaming connection. This document outlines the key patterns and pitfalls based on real-world production experience.

The #1 Rule: Never Block the Receive LoopCopy Anchor Link

The most critical mistake in WebSocket client implementations is performing slow or blocking operations directly inside the message handler (on_message).

Both the client and the server maintain receive and send buffers at the TCP/WebSocket level. Here's what happens when your on_message is slow:

1. Your client reads messages slowly → its receive buffer fills up
2. The server tries to send more messages but the client's buffer is full → server-side send buffer fills up
3. The server's message queue grows unbounded → backpressure propagates to the backend
4. Messages start getting dropped, connections become unstable, and ping/pong frames also get starved

This means slow message handling doesn't just hurt your client — it creates instability on the server side too, potentially affecting other clients sharing the same backend resources.

The examples below are using AWS; adapt to your infrastructure as needed.

❌ Bad Pattern

def on_message(ws, message):
    data = json.loads(message)
    # Slow I/O blocks the receive loop — buffers fill up on both sides
    kinesis_client.put_record(
        StreamName=config.KINESIS_STATS_STREAM,
        Data=json.dumps(data),
        PartitionKey=data['mac']
    )

✅ Good Pattern — Decouple with a Queue + Worker Thread

import queue
import threading
message_queue = queue.Queue()
def on_message(ws, message):
    # Fast: just enqueue, never block
    message_queue.put(message)
def worker():
    while True:
        message = message_queue.get(block=True)
        if message is None:
            continue
        data = json.loads(message)
        # Slow I/O happens here, in a separate thread
        kinesis_client.put_record(
            StreamName=config.KINESIS_STATS_STREAM,
            Data=json.dumps(data),
            PartitionKey=data['mac']
        )
# Start worker thread before connecting
threading.Thread(target=worker, daemon=True).start()

This keeps the WebSocket receive loop always free to handle ping/pong frames, regardless of how long your downstream processing takes.

Ping / Pong ConfigurationCopy Anchor Link

Your WebSocket connection needs regular keepalives to stay healthy. With websocket-client, ping_interval sends client-side pings and ping_timeout enforces how long to wait for a pong.

Recommended SettingsCopy Anchor Link

ws.run_forever(
    ping_interval=60,   # send a ping every 60 seconds
    ping_timeout=45     # wait up to 45 seconds for a pong response
)

Why Not More Aggressive?Copy Anchor Link

Using very short intervals (e.g., ping_interval=15, ping_timeout=10) seems safer but is actually more fragile under high message load — a busy receive loop may miss the pong window even on a healthy connection.

Handle Ping/Pong Events ExplicitlyCopy Anchor Link

def on_ping(ws, message):
    logger.info("Ping received")
def on_pong(ws, message):
    logger.info("Pong received — connection healthy")
ws = websocket.WebSocketApp(
    url,
    on_ping=on_ping,
    on_pong=on_pong,
    ...
)

Auto-ReconnectCopy Anchor Link

WebSocket connections will drop — due to network blips, server restarts, or deployments. Your script must reconnect automatically without human intervention.

def establish_ws():
    ws = websocket.WebSocketApp(
        'wss://api-ws.mist.com/api-ws/v1/stream',
        header=[f'Authorization: Token {API_TOKEN}'],
        on_open=on_open,
        on_message=on_message,
        on_error=on_error,
        on_close=on_close,
        on_ping=on_ping,
        on_pong=on_pong,
    )
    ws.run_forever(ping_interval=60, ping_timeout=45)
if __name__ == '__main__':
    while True:
        try:
            establish_ws()
        except Exception as e:
            logger.error(f"Connection error: {e}")
            logger.info("Reconnecting in 5 seconds...")
        time.sleep(5)  # back off before retrying

Important: Re-subscribe to your channel in on_open, not just on first connect — every reconnection needs a fresh subscription.

def on_open(ws):
    subscribe_message = {'subscribe': f'/sites/{SITE_ID}/stats/clients'}
    ws.send(json.dumps(subscribe_message))
    logger.info("Subscribed to channel")

Validate Your SubscriptionsCopy Anchor Link

The server sends an explicit failure event when a subscription cannot be completed:

{
  "event": "subscribe_failed",
  "channel": "/sites/aed1cb6f-xxxx-xxxx-xxxx-1bf541cfa4dd/stats/devices",
  "detail": "Server error, please try again later"
}

What to Expect from the ServerCopy Anchor Link

Every subscription attempt results in one of two responses:

• Success: { "event": "channel_subscribed", "channel": "/sites/aed1cb6f-xxxx-xxxx-xxxx-1bf541cfa4dd/stats/clients" }
• Failure: { "event": "subscribe_failed", "channel": "/sites/aed1cb6f-xxxx-xxxx-xxxx-1bf541cfa4dd/stats/devices", "detail": "..." }

How Your Client Should RespondCopy Anchor Link

• Track how many channels you requested vs how many returned channel_subscribed
• On receiving subscribe_failed, close and reconnect immediately — do not continue with a partially subscribed connection
• Add a watchdog timeout (e.g., 10 seconds after on_open) — if not all channels confirm in time, treat it as a partial failure and reconnect
• Log all subscription outcomes clearly so you can diagnose issues during live events

Secrets ManagementCopy Anchor Link

Never hardcode API tokens in your script. Use a secrets manager:

import boto3, json
def get_api_token(secret_name, region):
    client = boto3.client('secretsmanager', region_name=region)
    response = client.get_secret_value(SecretId=secret_name)
    return json.loads(response['SecretString'])['MIST_API_TOKEN']

Retrieve the token fresh on each establish_ws() call so token rotations are picked up automatically on reconnect.

Logging Best PracticesCopy Anchor Link

Good logging is essential for diagnosing issues during live events:

import logging, sys
logging.basicConfig(
    format='%(asctime)s.%(msecs)03d %(levelname)s: %(message)s',
    datefmt='%Y-%m-%d %H:%M:%S',
    level=logging.INFO,
    stream=sys.stdout,
    force=True
)
logger = logging.getLogger(__name__)

Log at key lifecycle points:

• WebSocket opened / closed
• Channel subscribed
• Ping sent / pong received
• Every N messages processed (e.g., every 100) to track throughput
• Any exception in message processing

Message Throttling & Queue MonitoringCopy Anchor Link

Under high load (thousands of concurrent clients), your queue can grow faster than it drains. Add basic monitoring:

THROTTLE_LOG_INTERVAL = 100
msgs_processed = 0
def worker():
    global msgs_processed
    while True:
        message = message_queue.get(block=True)
        process(message)
        msgs_processed += 1
        if msgs_processed % THROTTLE_LOG_INTERVAL == 0:
            logger.info(f"Processed {msgs_processed} messages. Queue size: {message_queue.qsize()}")

If queue size grows unbounded, your downstream sink (Kinesis, DB, etc.) is the bottleneck — consider batching or scaling your consumer.