Real-time data sync dashboard

⚡ New — Available in PR #737

Without real-time sync, your chatbot would only know about Shopify product changes the next time you ran a manual re-train. With real-time sync on, every product create, update, or delete in Shopify pushes through Hilal Chatbot’s webhook receivers and updates the chatbot’s knowledge automatically.

In this guide:

  • What real-time sync covers
  • How it works
  • Open the Sync Status panel
  • What each status means
  • Troubleshooting

What real-time sync covers (today)

Five Shopify webhook receivers are live:

ReceiverPathWhat it does
products/create/api/webhooks/receive/shopify/products-createNew product → enqueue indexing.
products/update/api/webhooks/receive/shopify/products-updateUpdated product → re-index.
products/delete/api/webhooks/receive/shopify/products-deleteDeleted product → remove from index.
collections/update/api/webhooks/receive/shopify/collections-updateCollection change → trigger re-evaluation.
app/uninstalled/api/webhooks/receive/shopify/app-uninstalledAuto-disable the integration backend-side.

Each receiver:

  1. Verifies Shopify’s HMAC signature against your SHOPIFY_CLIENT_SECRET.
  2. Deduplicates by webhook ID (in-memory; see Limitations).
  3. Forwards to Hilal Chatbot’s backend via signed fetch.
  4. Returns 200 to Shopify within 5 seconds (Shopify retries otherwise).

Other connected systems (Stripe, Zendesk) follow the same pattern; receivers ship in subsequent waves.

How it works (under the hood)

Shopify event
    ↓
HMAC-verified webhook receiver (Hilal Chatbot edge)
    ↓
Idempotency check
    ↓
Signed forward to Hilal backend
    ↓
Backend queue → selective re-train
    ↓
Chatbot knowledge updated

The receivers complete quickly (≤ 5s) so Shopify is happy; the backend handles the slow work asynchronously.

Step 1: Open the Sync Status panel

For Shopify deployments, the Sync Status panel lives on the store detail page in the Hilal Chatbot dashboard:

Settings → Integrations → Shopify → [your store]

Or for per-chatbot views:

Chatbot → Actions → Shopify → Sync status.

Real-time sync dashboard Screenshot: The Sync Status panel showing last-sync, last-event, and 24h event count.

The panel shows:

  • Last sync — when knowledge was last fully updated.
  • Last event — when the most recent Shopify webhook arrived.
  • Last error (if any) — most recent failure.
  • Live / Idle dot — green if events have been arriving, gray if quiet for ≥ 30 minutes.
  • 24-hour event counter — sanity check that webhooks are flowing.

Step 2: Read the indicators

IndicatorMeaning
🟢 Live, recent last-eventHealthy. Webhooks arriving, sync in flight.
Idle, no events for 30+ minEither nothing’s happening in the upstream system or webhooks aren’t reaching us.
🟢 Live, recent last-errorEvents arriving, but some are failing — investigate.
Idle, recent last-errorLikely a configuration issue (HMAC mismatch, backend offline).

Step 3: Configure receivers

If you used the standard Shopify install via Hilal Chatbot, receivers are auto-registered. For self-managed installs:

  1. Get the receiver URLs from the in-app Webhook URLs panel (or .env.example documentation).
  2. Register each in your Shopify Partners dashboard under your app’s Webhooks section.
  3. Verify HMAC signing uses your app’s client secret.

Limitations

  • Per-instance idempotency. The current dedupe is in-memory per Hilal Chatbot replica. In multi-replica production, the same event may briefly process twice across replicas — the backend’s durable dedupe absorbs this.
  • No widget push. “Real-time” here means upstream system → bot. Bot → widget push (live agent typing) uses SignalR, separate.
  • Stripe and Salesforce sync are tracked but not yet shipped here.

Troubleshooting

  • Sync shows Idle, but products are changing. HMAC mismatch — check SHOPIFY_CLIENT_SECRET matches the registered app. Or webhook URLs are wrong in Shopify Partners.
  • Last-error is recent. Open audit logs. Common causes: backend offline, signed-fetch clock skew, schema mismatch on a payload edge case.
  • Counter stuck at 0. Webhooks aren’t reaching us at all. Run a test by editing a product and watching the audit log; if no event arrives, Shopify’s webhook config is broken.

What’s next