# Best practices
# Important
The Geodir Platform web services are a collection of HTTP Interfaces to Geodir services that provide geographic data for your mapping applications. This guide describes techniques for configuring web service requests and parsing the responses. For specific Geofencing API documentation, see the **[Developer's Guide]
# What is a Web Service?
The Geodir Platform web services are an interface to request Geofencing API data from external services and use the data within your Geofencing applications. These services are designed to be used in conjunction with a map, subject to license restrictions in Geodir Terms of Service (opens new window).
Geofencing APIs web services use HTTP requests to specific URLs and pass URL parameters as arguments to the services. Typically, these services return data in the HTTP request as JSON or XML for parsing or processing by your application.
A typical Datacrime API web service request typically has the following format:
https://apis.geodir.co/geofencing/v1/responseFormat?parameters
where geofencing indicates the particular service requested and output indicates the response format (usually json or xml).
This guide describes some useful common practices for configuring your web service requests and processing your web service responses. See Developer's Guide (opens new window), for the full Geofencing API documentation.
Note: All Geofencing API applications require authentication. Learn more about Authentication Credentials (opens new window).
# SSL access
https://apis.geodir.co/geofencing/v1/*outputFormat?parameters*
HTTPS is required for all Geofencing API web service requests that contain user data or developer identifiers. Requests made over HTTP that include sensitive data may be rejected.
# Valid URL
You may think that a "Valid" URL is obviously blank, but that is not the case. A URL entered into an address bar in a browser, for example, may contain special characters (for example, "上海 + 中國"); the browser needs to internally translate those characters to a different encoding before transmission. Similarly, any code that generates or accepts UTF-8 input could treat URLs with UTF-8 characters as "valid", but would also need to translate those characters before sending them to a web server. This process is called URL Encoding (opens new window).
We need to translate special characters because all URLs must conform to the syntax specified by the W3 Uniform Resource Identifier (URI) (opens new window). In effect, this means that URLs must contain only a special subset of ASCII characters: the familiar alphanumeric symbols and some characters reserved for use as control characters within URLs. The following table summarizes these characters:
| Set | characters | URL usage |
|---|---|---|
| Alphanumeric | a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z 0 1 2 3 4 5 6 7 8 9 | Text strings, scheme usage (http), port (8080), etc. |
| Unreserved | - _ . ~ | Text strings |
| Reserved | ! * ' ( ) ; : @ & = + $ / ? % # [ ] | Control characters and/or Text Strings |
When creating a valid URL, you must ensure that it contains only the characters shown above. Conforming a URL to use this character set usually leads to two problems, one of omission and one of substitution:
- The characters you want to handle exist outside of the above set. For example, foreign language characters like 上海+中國 must be encoded using the above characters. By popular convention, spaces (which are not allowed within URLs) are often also represented by the plus '+' character.
- The characters exist within the above set as reserved characters, but must be used literally. For example, ? used within URLs to indicate the start of the query string; if you want to use the string "? Macchupicchu", you must encode the character '?'.
All characters to be URL-encoded are encoded with a '%' character and a two-character hexadecimal value corresponding to its UTF-8 character. For example, 上海+中國 in UTF-8 would be URL-encoded as %E4%B8%8A%E6%B5%B7%2B%E4%B8%AD%E5%9C%8B. Chain ? Macchupicchu would be URL-encoded as %3F+Macchupicchu.
Some common characters that need to be encoded are:
| Insecure Character | Encoded Value |
|---|---|
| Space | %20 |
| " | %22 |
| + | %2B |
| , | %2C |
| < | %3C |
| > | %3E |
| # | %23 |
| % | %25 |
| I | %7C |
Converting a URL that receives user input is sometimes tricky. For example, a user can enter an address like "Av Brasil&Bolivar". Generally, you should build your URL from its parts, treating any user input as character literals.
Also, URLs are limited to 8,000 characters for all web services. For most services, you will rarely go anywhere near this character limit. However, keep in mind that certain services have multiple parameters that can result in long URLs.
# Proper use of Geodir APIs
Poorly designed API clients can place more load than necessary on both the Internet and Geodir servers. This section contains some of the best practices for API clients. Following these best practices can help you prevent your application from being blocked for inadvertent API abuse.
# Exponential Regression
In rare cases, something may go wrong in handling your request; you may receive a 4XX or 5XX HTTP response code, or the TCP connection may simply fail somewhere between your client and the Geodir server. Often it is worth retrying the request, as the follow-up request may succeed when the original one failed. However, it is important not to simply repeat requests to Geodir's servers. This looping behavior can overload the network between your client and Geodir and cause problems for many parties.
A better approach is to retry with increasing delays between attempts. Typically, the delay increases by a multiplicative factor with each try, an approach known as Exponential Backoff (opens new window).
You should also be careful that there isn't a retry code higher in the application's call chain that leads to repeated requests in rapid succession.
# Synchronized Requests
A large number of synchronized requests to Geodir APIs can appear to be a Distributed Denial of Service (DDoS) attack on the Geodir infrastructure and be treated accordingly. To prevent this, you must ensure that API requests are not synchronized between clients.
For example, consider an application that displays the time in the current time zone. This application will probably set an alarm in the client's operating system and activate it at the start of the minute so that the displayed time can be updated. The application must not make any API calls as part of the processing associated with that alarm.
Making API calls in response to a fixed alarm is bad, as API calls are synchronized to the start of the minute, even across different devices, instead of evenly distributed over time. A poorly designed application that does this will spike traffic at sixty times normal levels at the start of every minute.
Instead, a good possible design is to have a second alarm set at a randomly chosen time. When this second alarm is triggered, the application calls the APIs it needs and stores the results. When the app wants to update its screen at the start of the minute, it uses the previously stored results instead of calling the API again. With this approach, API calls are evenly distributed over time. Also, API calls do not delay processing when the screen is updated.
Aside from the start of the minute, other common tone times to be careful not to set are at the start of an hour and at the start of each day at midnight.
# Processing Responses
Geodir's web services provide answers that are easy to understand, but not exactly easy to use. When performing a query, instead of displaying a set of data, you probably want to extract some specific values. Typically, you'll want to parse the responses from the web service and extract only the values that interest you.
The parsing schema you use depends on whether you are returning the output in XML or JSON. JSON responses, already in the form of Javascript objects, can be processed within blank Javascript on the client; XML responses must be processed using an XML processor and XML query language to address the elements within the XML format. We use XPath in the following examples as it is commonly supported in XML processing libraries.
# Processing XML with XPath
XML is a relatively mature structured information format used for data exchange. Although not as lightweight as JSON, XML provides more language support and more robust tools. The code for processing XML in Java, for example, is built into the javax.xml packages.
When processing XML responses, you should use an appropriate query language to select nodes within the XML document, rather than assuming that elements reside at absolute positions within the XML markup. XPath is a language syntax for uniquely describing nodes and elements within an XML document. XPath expressions allow you to identify specific content within the XML response document.
# XPath Expressions
Some familiarity with XPath goes a long way toward developing a solid parsing schema. This section will focus on how elements within an XML document are treated with XPath, which allows you to address multiple elements and build complex queries.
XPath uses expressions to select elements within an XML document, using a syntax similar to that used for directory paths. These expressions identify elements within an XML document tree, which is a hierarchical tree similar to that of a DOM. Typically, XPath expressions are greedy, indicating that they will match all nodes that match the supplied criteria.
We'll use the following abstract XML to illustrate our examples:
<WebServiceResponse>
<status>OK</status>
<result>
<type>sample</type>
<name>Sample XML</name>
<location>
<lat>37.4217550</lat>
<lng>-122.0846330</lng>
</location>
</result>
<result>
<message>The secret message</message>
</result>
</WebServiceResponse>
# Node Selection in Expressions
XPath selections select nodes. The root node spans the entire document. Select this node using the special expression "/". Note that the root node is not the top-level node of your XML document; in fact, it resides one level above and includes this top-level element.
Element nodes represent the various elements within the XML document tree. A WebServiceResponse element, for example, represents the top-level element returned in our sample service above. You can select individual nodes via absolute or relative paths, indicated by the presence or absence of a leading "/" character.
- Absolute path: The expression "/WebServiceResponse/result" selects all result nodes that are children of the WebServiceResponse node. (Note that both elements descend from the root node "/").
- Current context relative path: The expression "result" would match any result element within the current context. In general, you shouldn't have to worry about the context, since you typically render the results of the web service via a single expression.
Any of these expressions can be augmented by adding a wildcard path, indicated by a double slash ("//"). This wildcard indicates that zero or more elements can be matched in the intermediate path. The XPath expression "//name" for example, will match all nodes of that name in the current document.
By default, XPath expressions match all elements. You can restrict the expression to match a certain element by providing a predicate, which is enclosed in square brackets ([]). The XPath expression "/WebServiceResponse/result[2]" always returns the second result, eg.
Expression Type
| Type | Description |
|---|---|
| Root node | XPath Expression: "/" Selection: |
| Absolute Path | XPath Expression: "/WebServiceResponse/result" |
| Selection: | |
| Path with Wildcard | XPath Expression: "/WebServiceResponse//location" |
| Selection: | |
| Path with Predicate | XPath Expression: "/WebServiceResponse/result[2]/message" |
| Selection: | |
| All direct children of the first result | XPath Expression: "/WebServiceResponse/result[1]/*" |
| Selection: | |
| The name of a result whose type text is "sample." | XPath Expression: "/WebServiceResponse/result[type/text()='sample']/name" |
Selection: Sample XML|
It's important to note that when you select elements, you select nodes, not just the text inside those objects. Generally, you'll want to iterate over all matching nodes and extract the text. You can also match text nodes directly.
# Text Selection in Expressions
Text within an XML document is specified in XPath expressions using a text node operator. This "text()" operator indicates the extraction of text from the indicated node. For example, the XPath expression "//name/text()" will return all the text within the name elements.
Expression Type
| Type | Description |
|---|---|
| All text nodes (including whitespace) | XPath Expression: "//text()" Selection: sample Sample XML 37.4217550 -122.0846330 The secret message |
| Text Selection | XPath Expression: "/WebServiceRequest/result 2/message/text()" Selection: The secret message |
| Context Sensitive Selection | XPath Expression: "/WebServiceRequest/result type/text() = 'sample'/name/text()" Selection: Sample XML |
Alternatively, you can evaluate an expression and return a set of nodes and then iterate over that "set of nodes", extracting the text from each node. We use this approach in the following example.
For more information on XPath, see the W3C XPath Specification (opens new window).
# Processing JSON with JavaScript
JSON (Javascript Object Notation) has an obvious advantage over XML in that the response is lightweight. Parsing this result is trivial in JavaScript, since the format is already a valid JavaScript object. For example, to extract the value of the 'standard_address' keys inside a JSON result object, simply access them using the following code:
for (i = 0; i < myJSONResult.results.length; i++) {
myAddress[i] = myJSONResult.results[i].standard_address;
}
Note that because JSON can contain multiple values, it's wiser to iterate over the length of the results array if you want to capture all possible values. However, in practice, you may want to return only the first result (results[0]).
Parsing JSON in other languages is only moderately more difficult. The following Python example initiates a Geofencing web service request and displays all resulting standard_address values to the user within an array:
import json
import urllib.request
### The maps_key defined below isn't a valid Geodir API Key.
### You need to obtain your own API Key.
### See https://docs.geodir.co/geocoding/obtain-your-api-key
API_KEY = "YOUR_API_KEY"
GEOCODE_BASE_URL = "https://apis.geodir.co/geofencing/v1/json"
def geocode(address):
# Join the parts of the URL together into one string.
params = urllib.parse.urlencode({"address": address, "key": API_KEY,})
url = f"{GEOCODE_BASE_URL}?{params}"
result = json.load(urllib.request.urlopen(url))
if result["status"] in ["OK", "ZERO_RESULTS"]:
return result["results"]
raise Exception(result["message"])
if __name__ == "__main__":
results = geocode(address="Av+Arequipa")
print(json.dumps([s["standard_address"] for s in results], indent=2))
Output:
[
"Lima, Lima, Lima, Perú"
]