Forum Academy Marketplace Showcase Pricing Features

Using an Offset in API Connector to overome record/rate limits

I am querying a database service, which has a limit on the number of records returned (100 at a time only). If there are more records than 100, the response will contain an offset. To fetch the next page of records, we need to include offset in the next request’s parameters.

How can we achieve this using API Connector? Here is how one might do this in PHP using a while loop:

<?php
    // Initialize the offset.
    $offset = 0;

    // Initialize the result set.		
    $records = array();

    // Make calls to database, until all of the data has been retrieved...
    while (!is_null($offset)):
    	
    	// We're using an offset to get specific batches of records.
    	if ($offset > 0){
    		$http_params["offset"] = $offset;	
    	}
    	
    	if (isset($params["desiredrecord"])){
    		// Specifying desired record from BAAS
    		$http_params["desiredrecord"] = $params["desiredrecord"];
    	}

    	if (isset($params["limit"])){
    		// By default, the API will return 100 records per request.
    		// You can specify smaller batch sizes using the "limit"
    		// query parameter, as we are here.
    		$http_params["limit"] = $params["limit"];
    	}
    	
    	}

    	// Specify the URL to call.
    	$curlopt_url =   $this->BAAS_API_URL
    		           . $this->APP_ID
    		           . '/' . rawurlencode($endpoint)
    		           . "?" . http_build_query($http_params);

    	$args = array(
    	    'timeout'     => 30,
    	    'redirection' => 5,
    	    'blocking'    => $blocking,
    	    'headers'     => $http_headers,
    	    'cookies'     => array(),
    	    'body'        => null,
    	    'compress'    => false,
    	    'decompress'  => true,
    	    'sslverify'   => true
    	);

    	$response = wp_remote_get( $curlopt_url, $args );

    	// When getting a table, we'll build an array of records,
    	// when getting a record, we'll just return the record.
    	if (isset($body["records"])){
    		$records = array_merge($records,$body["records"]);	
    	} else {
    		$records = $body;
    	}

    	// Adjust the offset.
	// BAAS service returns NULL when the final batch of records has been returned.
	$offset = (isset($body["offset"]))? $body["offset"] : null;

endwhile;
?>