Secret Rotation

Secret rotation is the process of replacing webhook signing secrets with new ones. Hook Mesh supports zero-downtime rotation using a dual-secret period where both old and new secrets remain valid during the transition.

Why Rotate Secrets

Regular secret rotation is a critical security practice for several reasons:

Security hygiene: Limits exposure window if a secret is compromised
Compliance requirements: Many regulations mandate periodic credential rotation (SOC 2, PCI DSS, HIPAA)
Breach response: Quickly invalidate compromised secrets without service disruption
Employee offboarding: Rotate secrets when team members leave
Vendor changes: Update credentials after third-party integrations change

Best practice: Rotate webhook secrets every 90 days, or immediately after any suspected compromise.

Zero-Downtime Rotation Workflow

Hook Mesh uses a dual-secret period to enable rotation without downtime. Here's how it works:

Request rotation

Call the rotation API to generate a new secret

Dual-secret period begins

Both old and new secrets are valid for 24 hours

Update customer verification code

Deploy code that accepts both secrets during transition

Test with new secret

Send test webhooks to verify the new secret works

Old secret expires

After 24 hours, only the new secret remains valid

Zero downtime guaranteed: Webhooks signed with either secret are accepted during the 24-hour transition period.

Step-by-Step Rotation Guide

Step 1: Rotate the Secret

Call the rotation API endpoint to generate a new secret:

Rotate Secret API Call

curl -X POST https://api.hookmesh.com/v1/endpoints/ep_xyz/rotate-secret \
  -H "Authorization: Bearer ${HOOKMESH_API_KEY}"

Response

{
  "id": "ep_4Bz5Z0sR1tM7oP3xD",
  "secret": "whsec_x1y2z3a4b5c6d7e8f9g0h1i2j3k4l5m6n7o8p9q0r1s2t3u4v5w6",
  "secret_version": 2,
  "old_secret_expires_at": "2026-01-21T16:00:00Z"
}

Save the new secret! Store it securely - you'll need to provide it to your customer for verification.

Step 2: Update Verification Code

Update your customer's webhook handler to try both secrets during the transition period:

Node.js - Dual Secret Verification

const crypto = require('crypto');

const OLD_SECRET = process.env.HOOKMESH_WEBHOOK_SECRET_OLD;
const NEW_SECRET = process.env.HOOKMESH_WEBHOOK_SECRET_NEW;

function verifySignatureWithSecret(payload, headers, secret) {
  const webhookId = headers['webhook-id'];
  const timestamp = headers['webhook-timestamp'];
  const signature = headers['webhook-signature'];

  if (!webhookId || !timestamp || !signature) {
    return false;
  }

  // Extract signature (format: "v1,<signature>")
  const parts = signature.split(',');
  if (parts[0] !== 'v1') {
    return false;
  }
  const expectedSignature = parts[1];

  // Construct signed content
  const signedContent = `${webhookId}.${timestamp}.${JSON.stringify(payload)}`;

  // Compute HMAC
  const hmac = crypto.createHmac('sha256', secret);
  hmac.update(signedContent);
  const computedSignature = hmac.digest('base64');

  // Compare signatures (timing-safe)
  try {
    return crypto.timingSafeEqual(
      Buffer.from(expectedSignature),
      Buffer.from(computedSignature)
    );
  } catch (e) {
    return false;
  }
}

function verifyWebhook(payload, headers) {
  // Try new secret first (will be used after transition)
  if (verifySignatureWithSecret(payload, headers, NEW_SECRET)) {
    return true;
  }

  // Fall back to old secret during transition period
  if (verifySignatureWithSecret(payload, headers, OLD_SECRET)) {
    return true;
  }

  throw new Error('Invalid signature - neither secret matched');
}

// Express.js usage
app.post('/webhooks/hookmesh', express.json(), (req, res) => {
  try {
    verifyWebhook(req.body, req.headers);

    // Process webhook
    handleEvent(req.body);

    res.status(200).json({ received: true });
  } catch (error) {
    console.error('Webhook verification failed:', error.message);
    res.status(400).json({ error: error.message });
  }
});

Python - Dual Secret Verification

import hmac
import hashlib
import base64
import json
import os
from flask import Flask, request, jsonify

app = Flask(__name__)

OLD_SECRET = os.environ.get('HOOKMESH_WEBHOOK_SECRET_OLD')
NEW_SECRET = os.environ.get('HOOKMESH_WEBHOOK_SECRET_NEW')

def verify_signature_with_secret(payload, headers, secret):
    """Verify webhook signature with a specific secret."""
    webhook_id = headers.get('Webhook-Id')
    timestamp = headers.get('Webhook-Timestamp')
    signature = headers.get('Webhook-Signature')

    if not all([webhook_id, timestamp, signature]):
        return False

    # Extract signature (format: "v1,<signature>")
    parts = signature.split(',')
    if len(parts) != 2 or parts[0] != 'v1':
        return False
    expected_signature = parts[1]

    # Construct signed content
    signed_content = f"{webhook_id}.{timestamp}.{json.dumps(payload, separators=(',', ':'))}"

    # Compute HMAC
    computed_signature = base64.b64encode(
        hmac.new(
            secret.encode('utf-8'),
            signed_content.encode('utf-8'),
            hashlib.sha256
        ).digest()
    ).decode('utf-8')

    # Compare signatures (timing-safe)
    return hmac.compare_digest(expected_signature, computed_signature)

def verify_webhook(payload, headers):
    """Verify webhook with either old or new secret."""
    # Try new secret first (will be used after transition)
    if verify_signature_with_secret(payload, headers, NEW_SECRET):
        return True

    # Fall back to old secret during transition period
    if verify_signature_with_secret(payload, headers, OLD_SECRET):
        return True

    raise ValueError('Invalid signature - neither secret matched')

@app.route('/webhooks/hookmesh', methods=['POST'])
def handle_webhook():
    try:
        verify_webhook(request.json, request.headers)

        # Process webhook
        handle_event(request.json)

        return jsonify({'received': True}), 200
    except ValueError as e:
        print(f'Webhook verification failed: {str(e)}')
        return jsonify({'error': str(e)}), 400

def handle_event(payload):
    # Your webhook processing logic here
    print(f'Processing webhook: {payload}')

if __name__ == '__main__':
    app.run(port=3000)

Step 3: Test the New Secret

Before the old secret expires, send test webhooks to verify the new secret works:

Send Test Webhook

curl -X POST https://api.hookmesh.com/v1/endpoints/ep_xyz/test \
  -H "Authorization: Bearer ${HOOKMESH_API_KEY}" \
  -H "Content-Type: application/json" \
  -d '{
    "payload": {
      "type": "test",
      "message": "Testing new secret after rotation"
    }
  }'

Check your logs to confirm the webhook was verified successfully with the new secret.

Step 4: Remove Old Secret (After 24 Hours)

Once the old secret expires, update your code to only use the new secret:

Single Secret Verification (After Rotation)

const crypto = require('crypto');

const SECRET = process.env.HOOKMESH_WEBHOOK_SECRET; // Only new secret

function verifyWebhook(payload, headers, secret) {
  const webhookId = headers['webhook-id'];
  const timestamp = headers['webhook-timestamp'];
  const signature = headers['webhook-signature'];

  if (!webhookId || !timestamp || !signature) {
    throw new Error('Missing required headers');
  }

  // Extract signature (format: "v1,<signature>")
  const parts = signature.split(',');
  if (parts[0] !== 'v1') {
    throw new Error('Invalid signature version');
  }
  const expectedSignature = parts[1];

  // Construct signed content
  const signedContent = `${webhookId}.${timestamp}.${JSON.stringify(payload)}`;

  // Compute HMAC
  const hmac = crypto.createHmac('sha256', secret);
  hmac.update(signedContent);
  const computedSignature = hmac.digest('base64');

  // Compare signatures (timing-safe)
  if (!crypto.timingSafeEqual(
    Buffer.from(expectedSignature),
    Buffer.from(computedSignature)
  )) {
    throw new Error('Invalid signature');
  }

  return true;
}

// Express.js usage
app.post('/webhooks/hookmesh', express.json(), (req, res) => {
  try {
    verifyWebhook(req.body, req.headers, SECRET);

    // Process webhook
    handleEvent(req.body);

    res.status(200).json({ received: true });
  } catch (error) {
    console.error('Webhook verification failed:', error.message);
    res.status(400).json({ error: error.message });
  }
});

Automatic Expiration Strategy

You can automate secret rotation by scheduling regular rotations:

Automated 90-Day Rotation

const cron = require('node-cron');

// Schedule rotation every 90 days
cron.schedule('0 0 1 */3 *', async () => {
  console.log('Starting scheduled secret rotation...');

  // Get all endpoints
  const endpoints = await fetch(
    'https://api.hookmesh.com/v1/endpoints?application_id=app_xyz',
    {
      headers: { 'Authorization': `Bearer ${process.env.HOOKMESH_API_KEY}` }
    }
  ).then(r => r.json());

  // Rotate each endpoint's secret
  for (const endpoint of endpoints.data) {
    try {
      const response = await fetch(
        `https://api.hookmesh.com/v1/endpoints/${endpoint.id}/rotate-secret`,
        {
          method: 'POST',
          headers: { 'Authorization': `Bearer ${process.env.HOOKMESH_API_KEY}` }
        }
      );

      const { secret, old_secret_expires_at } = await response.json();

      // Store new secret in your secret manager
      await storeSecret(endpoint.id, secret);

      // Notify customer of rotation
      await notifyCustomer(endpoint.id, {
        newSecret: secret,
        expiresAt: old_secret_expires_at,
        message: 'Your webhook secret has been rotated for security. Update your verification code within 24 hours.'
      });

      console.log(`Rotated secret for endpoint ${endpoint.id}`);
    } catch (error) {
      console.error(`Failed to rotate secret for ${endpoint.id}:`, error);
    }
  }
});

async function storeSecret(endpointId, secret) {
  // Store in your secret manager (AWS Secrets Manager, HashiCorp Vault, etc.)
  // Example: await secretsManager.updateSecret(endpointId, secret);
}

async function notifyCustomer(endpointId, rotationInfo) {
  // Send email or notification to customer
  // Example: await sendEmail(customer.email, rotationInfo);
}

Manual Rotation vs Automatic Rotation

Approach	When to Use	Pros	Cons
`Manual`	Breach response, employee offboarding	Full control, immediate action	Requires human intervention
`Automatic`	Regular hygiene, compliance	Consistent, no manual work	Requires coordination with customers

Best Practices

Rotate every 90 days: Regular rotation limits exposure window and meets compliance requirements
Use the dual-secret period: Always deploy code that accepts both secrets before the old one expires
Test before expiration: Send test webhooks to verify the new secret works before the transition period ends
Maintain audit trails: Log all secret rotations with timestamps and reasons
Store secrets securely: Use environment variables or secret managers (AWS Secrets Manager, HashiCorp Vault)
Notify customers: Alert customers before automated rotations so they can prepare
Emergency rotation plan: Document the process for immediate rotation in case of breach
Monitor verification failures: Spike in failures may indicate rotation issues

Emergency Rotation

If you suspect a secret has been compromised, rotate immediately:

Emergency Rotation Script

#!/bin/bash
# emergency-rotate.sh - Rotate all endpoint secrets immediately

ENDPOINT_ID="ep_xyz"
API_KEY="${HOOKMESH_API_KEY}"

echo "Starting emergency rotation for endpoint ${ENDPOINT_ID}..."

# Rotate the secret
RESPONSE=$(curl -s -X POST "https://api.hookmesh.com/v1/endpoints/${ENDPOINT_ID}/rotate-secret" \
  -H "Authorization: Bearer ${API_KEY}")

NEW_SECRET=$(echo ${RESPONSE} | jq -r '.secret')
EXPIRES_AT=$(echo ${RESPONSE} | jq -r '.old_secret_expires_at')

echo "New secret: ${NEW_SECRET}"
echo "Old secret expires at: ${EXPIRES_AT}"

# Update environment variables
echo "HOOKMESH_WEBHOOK_SECRET_NEW=${NEW_SECRET}" >> .env.new
echo ""
echo "Emergency rotation complete!"
echo "1. Deploy updated .env.new file immediately"
echo "2. Update verification code to use both secrets"
echo "3. Test with: curl -X POST https://api.hookmesh.com/v1/endpoints/${ENDPOINT_ID}/test"
echo "4. Monitor logs for verification failures"

Act fast: In a breach scenario, you have 24 hours to update customer code before the old secret expires. Coordinate with customers immediately.

Monitoring Rotation Health

Track rotation metrics to ensure smooth transitions:

Rotation Health Check

async function checkRotationHealth() {
  const endpoints = await fetch(
    'https://api.hookmesh.com/v1/endpoints?application_id=app_xyz',
    {
      headers: { 'Authorization': `Bearer ${process.env.HOOKMESH_API_KEY}` }
    }
  ).then(r => r.json());

  const metrics = {
    total: endpoints.data.length,
    recentRotations: 0,
    oldSecrets: 0,
    inTransition: 0
  };

  const now = new Date();
  const ninetyDaysAgo = new Date(now.getTime() - 90 * 24 * 60 * 60 * 1000);

  for (const endpoint of endpoints.data) {
    const secretAge = new Date(endpoint.secret_updated_at);

    // Count recent rotations (within 7 days)
    if (now - secretAge < 7 * 24 * 60 * 60 * 1000) {
      metrics.recentRotations++;
    }

    // Count secrets older than 90 days
    if (secretAge < ninetyDaysAgo) {
      metrics.oldSecrets++;
    }

    // Check if currently in transition period
    if (endpoint.old_secret_expires_at) {
      const expiresAt = new Date(endpoint.old_secret_expires_at);
      if (expiresAt > now) {
        metrics.inTransition++;
      }
    }
  }

  console.log('Rotation Health Metrics:', metrics);

  // Alert if many secrets are old
  if (metrics.oldSecrets > metrics.total * 0.2) {
    console.warn(`Warning: ${metrics.oldSecrets} endpoints have secrets older than 90 days`);
  }

  return metrics;
}

// Run daily health check
setInterval(checkRotationHealth, 24 * 60 * 60 * 1000);