- Your server must be able to receive
HTTP/1.1requests and support keep-alive requests.
- Your server needs to respond to
HTTP GETrequests on your webhook URL for verification.
- Your server needs to respond to
HTTP POSTrequests on your webhook URL to acknowledge receiving notifications.
Put bluntly: your app is not the only one powered by Layer. If your server gets slow, we can’t allow it to impact our other customers. For this reason, we impose very strict latency requirements on your response to our webhooks.
Your server must respond within one second.
Or else what? Ideally this never applies to your webhook. That said, if your server takes longer than one second to respond for some reason, that webhook call gets queued for a 10-second backoff, after which it will be retried. If the retry succeeds (your server responds within one second), all is well, end of story. If not, we back off a bit longer and retry yet again. And so on. There’s a limit, of course, since we can’t conceivably allow slow apps to create an unbounded backlog in our system. The backoff delays are roughly:
- 10 seconds
- 30 seconds
- 120 seconds
- 300 seconds
- 1200 seconds
This amounts to about 28 minutes total, which should give you plenty of flexibility in terms of doing releases, server maintenance, etc. It’s important to note that this sequence of backoffs and retries applies to every single webhook call that times out (or fails). If you have just one outlier, other webhook calls will continue to flow normally while that one call is backing off and being retried.
If a single webhook call times out (or fails) after all of these backoffs and retries, the call will be dropped (along with any other calls currently backing off) and your webhook will be deactivated. If this happens, you will receive an email notification alerting you to this condition. While your webhook is deactivated, no traffic will attempt to flow to your server. It’s up to you to remedy the issue and reactivate your webhook, at which point traffic will begin flowing again.
It’s worth noting that if a webhook call occasionally times out (or fails), then succeeds upon retry, the event for that webhook call can arrive out of order. You should factor this consideration into the design of your integration.
With just one second to respond, don’t waste that time forcing us to open a connection to your server. Simply establishing a connection can take hundreds of milliseconds. Doing that on every call is obviously prohibitively expensive latency-wise, which is why we require that your server supports HTTP/1.1 with persistent connections (keep-alive).
To maximize throughput, we encourage you to configure your server with as liberal settings as possible. Many web servers allow you to control things like request timeout and maximum number of requests per connection. The closer to infinity you can set these values, the less often Layer is forced to establish a new connection, and the lower the end-to-end latency will be.
If your app sees lots of traffic, request timeout may not be as relevant, since we’ll be hitting your server frequently enough for it not to kick in. During testing, however, or with low-traffic apps, please be aware of it. If requests time out too soon, forcing us to establish a connection could become a major source of latency, and ultimately a potential cause for webhook deactivation.
If you find your webhook getting deactivated unexpectedly, suspect these configurables as potential factors.
The more messaging traffic you pump through Layer, the more traffic your webhook server needs to handle. Given the strict latency requirement, we highly encourage you to leverage a queueing system such as Amazon SQS, Google Cloud Pub/Sub, or something like RabbitMQ or Kafka, depending upon your requirements. By simply “dumping” a webhook into a queue and consuming it in a separate process, you can keep webhook response latency to a minimum.