Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Table of Contents

This wiki captures standards, best practices and guidance compiled from industry's thought leaders for developing RESTful APIs. This guidance is intended to provide project teams with information that will result in the development of APIs that conform to a common style and that present service consumers with familiar paradigms across services developed by multiple project teams. There are no mandatory components to this guidance but readers are encouraged to consider the implications of not following this guidance, e.g. making it harder for API consumers to use their services, requiring custom integration code to deal with new data formats and causing unnecessary API versioning issues.

...

  1. A resource is an abstraction of information or a concept. Choose a URI that incorporates the resource name to identify the information or concept. Resource names should be nouns. For example, the SDP Vocabulary Service includes the concept of a Survey to define data collection instruments. The resource name should include "survey" to indicate its meaning, e.g. a resource that provides a list of surveys could be https://sdp-v.services.cdc.gov/api/surveys  
  2. Use plural nouns for consistency. For example, https://sdp-v.services.cdc.gov/api/surveys and https://sdp-v.services.cdc.gov/surveys/S-11 
  3. Do not encode API version numbers in the URL. Cool URIs don't change, see the section on versioning for more information.
  4. Support content negotiation (selection of the representation format - XML, JSON, etc.) via the HTTP Accept header. For convenience it is also acceptable to additionally support URL-based format specification, e.g. http://example.org/docs/1.xml to specify an XML representation of the http://example.org/docs/1 resource.
  5. Do not include operation or action names in the URL, e.g. http://www.acme.com/products/new. While this style of URI is common in Web applications, where the example URI might identify a Web form that aids in creation of a new product, Web APIs should instead use URIs like http://www.acme.com/products in conjunction with a HTTP POST that includes a representation of the new resource to be created.
  6. Filtering, sorting, searching, pagination and resource versions (not API versions) should be specified using query parameters. For example, https://sdp-v.services.cdc.gov/api/surveys?limit=100 where the limit query parameter is used to limit the number of items returned. This allows multiple such qualifiers to be used together without requiring complex hierarchies within the URI.

Examples of Bad URLs

  • https://www.acme.com/product (singular product)
  • https://www.acme.com/products/filter/cats (encodes qualifier in URI path)
  • https://www.acme.com/product/1234 (singular product)
  • https://www.acme.com/photos/products/new (encodes operation name in URI)
  • https://www.acme.com/api/v1/products (encodes API version in URI)

Examples of Good URLs

  • https://www.acme.com/products (plural product)
  • https://www.acme.com/products/?filter=cats (encodes qualifier in URI query parameter)
  • https://www.acme.com/products/1234 (plural product)
  • https://www.acme.com/photos/products (in conjunction with POSTing a representation of the resource to be created)
  • https://www.acme.com/products (do not encode API version in URI, instead follow guidance in the section on versioning)

Representations

Representations encode the actual or intended state of a resource at a point in time. The state of a resource can change over time and it can be captured in different formats (e.g. XML or JSON). This means that a resource can have multiple representations that differ over time and differ in format. Representation metadata is included in HTTP headers and is used for various purposes by both clients and servers (e.g. determining the format).

...

  • The data format, e.g. XML or JSON
  • The specific dialect of the data format, e.g. Atom Feed or Entry or a particular XML or JSON schema as a subset of the generic format specified above
  • The use of hypertext within the format, in particular any link relationships used

Registering custom media types is encouraged; registration under the vendor subtree (application/vnd.xyz, e.g. application/vnd.gov.cdc.sdp.surveys+xml) provides a lightweight framework for registering a new media type.

...

Media types can include parameters and it is recommended that the version of a media type is specified using a parameter (by convention the letter v), e.g. application/vnd.gov.cdc.sdp.surveys+xml;v=2.0, for further details see the guidance in the section on versioning.

XML vs JSON

The two most common data formats used for RESTful Web services are Extensible Markup Language (XML) and JavaScript Object Notation (JSON). Each has advantages and disadvantages and choosing between them requires careful consideration of the use case and capabilities of each format.

...

HTTP methodOperationIdempotentExampleNotes
GETRetrieve a representation of a resource. Resource can be a collection or single itemYes

All the surveys in a vocabulary Service can be retrieved by:

GET https://sdp-v.services.cdc.gov/surveys 

Filtering can be controlled using query parameters. 

Control metadata should be used to optimize requests, e.g. If-None-Match to only retrieve a representation if it has changed since the last time the client retrieved it.
POSTCreate or perform an operation on the resourceNo

A new survey can be created using Vocabulary Service by 

POST https://sdp-v.services.cdc.gov/surveys 

The new survey URI will be assigned automatically

The initial state of the new resource should be included in the request.
PUTUpdate a resource by replacing its state entirely with that conveyed by the supplied representation Yes

A survey can be updated using Vocabulary Service by 

PUT https://sdp-v.services.cdc.gov/surveys/{id}

Full representation of the object should be sent in the request. 

Control metadata should be used to perform optimistic locking. e.g, use an If-Match header with an etag to avoid overwriting updates from others.

PATCHUpdate part of resource using the delta conveyed by the supplied representationNo

A survey could be updated using the Vocabulary service by

PATCH https://sdp-v.services.cdc.gov/surveys/{id}

A delta from the current state to the desired state should be sent in the request.

Control metadata should be used to perform optimistic locking, e.g. If-Match to ensure that the resource has not changed since the delta was created.

DELETEDelete a resourceYes

A survey can be deleted using Vocabulary Service by 

DELETE https://sdp-v.services.cdc.gov/surveys/{id}

Control metadata should be used to perform optimistic locking, e.g. If-Match to ensure that the resource being deleted has not changed since the last time it was retrieved. 
OPTIONSObtain the supported methods and CORS on a resourceYes

Supported methods and control information can be retrieved by 

OPTIONS https://sdp-v.services.cdc.gov/surveys 

Some browsers use this method to support CORS by querying the resource for Access-Control-Allow-Origin header 



Control Data

Control data defines the purpose of a message between components, such as the action being requested or the meaning of a response. For example, cache behavior can be modified by control data specified as HTTP Cache-Control header included in the request or response message. Etags are useful to help prevent simultaneous updates (optimistic locking) of a resource from overwriting each other. Control data using HTTP headers should be used appropriately. Below is an example of control data in a Request and Response

Request

GET /surveys HTTP/1.1
Host: sdp-v.services.cdc.gov
If-None-Match: "0eaac798-cb6a-41fd-ac7a-78f1a40945c6"

...

  • 301 Move Permanently indicates that the URI of the requested resource has been changed and the new URI is provided in the Location header.
  • 302 Found indicates that the resource is temporarily identified by a different URI (supplied in the Location header). New changes in the URI might be made in the future. Therefore, this same URI should be used by the client in future requests.
  • 304 Not Modified indicates that request preconditions (e.g. If-Modified-Since or If-None-Match) were not met or the client has the response already in its cache. In both cases there is no need to transfer the same representation again. This response should be used for requests with GET or HEAD methods.

4xx (Client Error Category)

...

  • 400 Bad Request indicates that the request by the client was not processed, as the server could not understand what the client is asking for.
  • 401 Unauthorized indicates that the client is not allowed to access resources, and should re-request with the required credentials.
  • 403 Forbidden indicates that the request is valid and the client is authenticated, but the client is not allowed to access the page or resource for any reason. E.g sometimes the authorized client is not allowed to access the directory on the server.
  • 404 Not Found indicates that the requested resource is not available now.
  • 406 Not Acceptable indicates that the server was unable to return a representation of the resource in the format requested by the client (in the request Accept header).
  • 410 Gone indicates that the requested resource is no longer available which has been intentionally moved.
  • 412 Precondition Failed indicates that access to the target resource has been denied. This happens with conditional requests on methods other than GET or HEAD when the condition defined by the If-Unmodified-Since or If-None-Match headers is not fulfilled. In that case, the request, usually an upload or a modification of a resource, cannot be made and this error response is sent back.
  • 415 Unsupported Media Type indicates the media format of the requested data is not supported by the server, so the server is rejecting the request.

...

Hypermedia As The Engine Of Application State (HATEOAS)

HATEOAS introduces discoverability and evolvability, providing a way of making a RESTful API self-documenting. Rather than coding a client to use specific service URIs, it instead relies on a generic understanding of hypermedia and a detailed knowledge of the representation format in use and the link relationships used in that format.

The client enters a RESTful HATEOAS API through a simple fixed or previously bookmarked URL. All future actions the client may take are discovered within resource representations or HTTP headers returned from the server. The media types used for these representations, the link relations they may contain, and operations available on the resource are standardized. The client transitions through application states by selecting from the links within a representation or by manipulating the representation in other ways afforded by its media type. In this way, RESTful interaction is driven by hypermedia, rather than out-of-band information.

Benefits of HATEOAS include:

  1. The client is informed of what actions can be taken using the contents of responses
  2. The server can change its URI scheme without breaking clients, provided clients behave according to the expectations of HATEOAS
  3. New capabilities can be advertised by putting new links in the response

Guidelines:

  1. Use links to allow clients to discover locations and operations
  2. Use Hypermedia Link Relations to specify the relationship
  3. Use the atom:link structure to capture links in XML representations
  4. Use JSON HAL to capture links in JSON representations

...

Code Block
languagetext
Strict-Transport-Security: max-age=31536000; includeSubDomains

Caching

HTTP Caching eases the load on a service since it doesn’t need to serve all clients directly, and it can improve performance if it takes less time to transmit the resource back to the client. Caching is use case dependent. Every request and response can define its own caching policies via the Cache-Control HTTP header. For requests, the directive specifies whether cached responses are permitted and under what circumstances. For responses, the directive specifies whether it can be cached, under what conditions and for how long. Below is a list of Cache-Control directive that can be used in HTTP requests and responses.

...

When a client maintains its own cache of information, use of eTags as described above can also provide a similar benefit. E.g. a client can include an If-None-Match header to make a request conditional upon a different representation of the resource being available. Note that eTags serve to shortcut requests that would return the same data as a previous request whereas caching serves to distribute a time-limited response to intermediate servers that can offload request processing from the source of truth. eTags work well to reduce the number of duplicate requests from a single client, caches work well to share the result of one client's request with other subsequent clients.

...

Compatibility is an important consideration since clients and servers are often the responsibility of different parties, there are often many more clients of an API than providers, and it can be difficult to upgrade all parties at the same time. The rest of this section describes approaches to versioning of APIs that follows Postel's Law. Additional material on application of Postel's law to the design of APIs is available in Jose Luis Ordiales's blog.

Versioning RESTful APIs

A RESTful API's contract is defined by the constraints specified in the Uniform Interface. The Uniform Interface comprises of these four constraints

  1. Identification of resources or resource identifier
  2. Manipulation through representations
  3. Self-descriptive messages
  4. Hypermedia as the engine of application state (HATEOAS)

The following guidelines focus on resource identifier (URL) and representation (XML, JSON, HTML, etc) versioning and these are from Howard Dierking's blog:

...

2. Media Type Versioning: When consuming a RESTful API a client specifies the format of content that it expects as part of HTTP GET request using the Accept header. The client can specify XML, JSON, or some other format that it expects in the body of the response. Additionally, the Accept header can also be used to specify a custom media type that allows the client application to specify the version of the resource representation it expects in the response. For a breaking change to the survey representation, this approach is shown below.

Version 1: 

Code Block
languagejs
Request:
GET /surveys/1 HTTP/1.1
Accept: application/vnd.sdp-v-services+json;v=1.0

Response:
HTTP/1.1 200 OK
Content-Type:  application/vnd.sdp-v-services+json;v=1.0; charset=utf-8 
{"id":1, "name":"Salmonella outbreak","creator":"John Doe"}

Version 2:

Code Block
languagejs
Request: 
GET /surveys/1 HTTP/1.1 
Accept: application/vnd.sdp-v-services+json;v=2.0

Response: 
HTTP/1.1 200 OK 
Content-Type:  application/vnd.sdp-v-services+json;v=2.0; charset=utf-8 
{"id":1, "name":"Salmonella outbreak","creator":{"firstname":"John", "lastname": "Doe"}}

...

It is recommended that APIs that require authentication and authorization utilize OAuth for that purpose. Use of OAuth to secure CDC APIs is detailed in Using OAuth to Secure CDC APIs.

API Definition

APIs need to be described using a standard, language-agnostic interface description which allows both humans and computers to discover and understand the capabilities of the service without access to source code, documentation, or through network traffic inspection. Consumers of an API should be able understand and interact with the service with a minimal amount of implementation logic. The OpenAPI Specification is the industry standard for defining and describing RESTful APIs. APIs should be defined and described using the latest version of OpenAPI Specification.

...

Note that there is currently a divergence between JSON Schema and Open API, full details and a proposed approach to work around the incompatibility is described in this article by Phil Sturgeon.

Advertising API Availability

In addition to creating an API definition and offering it at the standard endpoint as described above, it is recommended that consideration be given to advertising API availability via API registries. CDC currently hosts two such registries:

References

  1. Fielding's dissertation on REST
  2. Richardson Maturity Model
  3. Designing HTTP Interfaces and RESTful Web Services
  4. Securing Microservices APIs, Sustainable and Scalable Access Control - Matt McLarty, Rob Wilson & Scott Morrison
  5. Hypermedia Link Relations
  6. Media Types
  7. HTTP resources and specifications
  8. HTTP Status codes
  9. HTTP Caching