Rate limit handling

[andre-vm]

I need to update a contact's segments at once. Given that there's no request that allows me to set them all at once, I'm forced to call resend.Contacts.Segments.add(params) and resend.Contacts.Segments.remove(params) multiple times in a row, which will often cause a RateLimitError.

It would be nice to read the retry-after attribute from the response header in order to know how much I'll have to wait before retrying. But currently the SDK doesn't expose any of the header attributes in the exception object.

I see that #179 goes well in the direction of adding the response header to the returned objects. It would be nice have it added to the ResendError class as well.

I know this is a lot to ask, but even better would be to have the retry logic embedded in the SDK itself as an optional feature. The SDK could track the ratelimit-remaining and ratelimit-reset attributes from the responses of previously successful requests, and then avoid the rate limit error altogether by waiting in advance when the user runs out of remaining requests within the current window.

[andre-vm]

Actually, this problem can be nicely solved by using a custom HTTP client. It's possible to use an HTTP adapter that respects the retry-after interval in case of error 429. The custom client I came up with also tracks the ratelimit-remaining and ratelimit-reset to avoid that error altogether, as I had suggested.

import time
from dataclasses import dataclass
from http import HTTPMethod
from typing import Collection, Dict, List, Mapping, Optional, Self, Tuple, Union

import requests
import resend
from requests.adapters import HTTPAdapter
from requests.structures import CaseInsensitiveDict
from urllib3.util.retry import Retry


@dataclass(frozen=True)
class RateLimitState:
    limit: int = 0
    remaining: int = 0
    next_reset: float = 0.0
    next_retry: float = 0.0

    @property
    def wait_seconds(self) -> float:
        return self.get_wait_seconds()

    def get_wait_seconds(self, now: Optional[float] = None) -> float:
        now = now or time.time()

        if self.next_retry > now:
            return self.next_retry - now

        if self.remaining == 0 and self.next_reset > now:
            return self.next_reset - now

        return 0

    @classmethod
    def from_headers(cls, headers: CaseInsensitiveDict, now: Optional[float] = None) -> Self:
        limit = int(headers.get("ratelimit-limit", "0"))  # Maximum number of requests allowed within a window.
        remaining = int(headers.get("ratelimit-remaining", "0"))  # How many requests you have left within the current window.
        reset = float(headers.get("ratelimit-reset", "0"))  # How many seconds until the limits are reset.
        retry_after = float(headers.get("retry-after", "0"))  # How many seconds you should wait before making a follow-up request.

        now = now or time.time()
        next_reset = now + reset
        next_retry = now + retry_after

        return cls(
            limit=limit,
            remaining=remaining,
            next_reset=next_reset,
            next_retry=next_retry,
        )


class ResendCustomClient(resend.HTTPClient):
    timeout: int
    session: requests.Session
    rate_limit_state: RateLimitState

    def __init__(
        self,
        timeout: int = 30,
        session: Optional[requests.Session] = None,
    ):
        self.timeout = timeout
        self.session = self._http_retry_session(session=session)
        self.rate_limit_state = RateLimitState()

    def request(
        self,
        method: str,
        url: str,
        headers: Mapping[str, str],
        json: Optional[Union[Dict[str, object], List[object]]] = None,
    ) -> Tuple[bytes, int, Mapping[str, str]]:
        wait_seconds = self.rate_limit_state.wait_seconds
        if wait_seconds:
            time.sleep(wait_seconds)

        try:
            response = self.session.request(
                method=method,
                url=url,
                headers=headers,
                json=json,
                timeout=self.timeout,
            )
            self.rate_limit_state = RateLimitState.from_headers(response.headers)
            return response.content, response.status_code, response.headers

        except requests.RequestException as e:
            # This gets caught by the request.perform() method
            # and raises a ResendError with the error type "HttpClientError"
            if e.response:
                self.rate_limit_state = RateLimitState.from_headers(e.response.headers)

            raise RuntimeError(f"Request failed: {e}") from e
        

    @staticmethod
    def _http_retry_session(
        retries: int = 3,
        backoff_factor: float = 0.3,
        status_forcelist: Collection[int] = (500, 502, 503, 504),
        session: Optional[requests.Session] = None,
    ):
        session = session or requests.Session()
        retry = Retry(
            total=retries,
            read=retries,
            connect=retries,
            backoff_factor=backoff_factor,
            status_forcelist=status_forcelist,
            respect_retry_after_header=True,
            allowed_methods=[
                HTTPMethod.HEAD,
                HTTPMethod.TRACE,
                HTTPMethod.GET,
                HTTPMethod.PUT,
                HTTPMethod.OPTIONS,
                HTTPMethod.DELETE,
                HTTPMethod.POST,
            ],
        )
        adapter = HTTPAdapter(max_retries=retry)
        session.mount("http://", adapter)
        session.mount("https://", adapter)
        return session


resend.default_http_client = ResendCustomClient()

[drish]

@andre-vm thanks for opening this issue and sharing your use case, I will prioritize getting the header data exposed on responses as well as in ResendError so that would simplify things for you. As for the automatic retry on errors: there are some pros and cons to automatically retrying errors on the SDK level - I will bring that up with the core team

[hasancemcerit]

@drish it seems that the PR #179 has been merged, however, when I update the package, I can't seem to get access to the newly exposed rate limit headers.
Am I doing something wrong here?

[drish]

hey! I have just released 2.23.0 which should have the rate limit headers, let me know if anything comes up!
thanks @andre-vm @hasancemcerit