An introduction to service APIs

By Alistair Stead

18 March 2014 Technology engineering

‎

In this post we discuss service APIs – a lightweight, pragmatic approach to web services, with multiple use cases.

Executive summary

Every company starts with a single website or web application, but as a company’s internet presence grows, many different applications and websites are deployed. With a traditional approach of treating each of these applications as a completely separate solution, a number of problems occur: common functionality is implemented inconsistently across applications, maintenance costs are high and time to market for new applications suffers.

By introducing a service API the problems of the traditional approach can be overcome. Many applications can reuse a set of common functionality that is implemented only once into the service layer, and then reused by individual applications and websites. This approach leads to consistent applications, lower maintenance costs and a much improved time to market.

Service APIs also help integrate third party applications in a consistent and robust way, and can help workaround performance limitations if they exist in third party applications. Creating multichannel applications (exposing functionality not just via the web browser, but via iPhone apps, for example) becomes easier when the functionality is implemented in the service API.

To migrate to a service API is not a trivial task, and a well-defined process is advisable. The recommended starting point is a functional analysis of what the goals of the API are. Next, a technical design is created to define the service APIs architecture. Finally, the API is developed and tested. Because APIs will be used by other application developers, documentation of the API is crucial and should be part of the development process.

Migrating from a non-service based to a service-based environment is achievable. It is possible even to migrate feature by feature, so a full rewrite or re-architecture of the application is avoided.

Introduction

Web services have been around for a while. Terms like ‘Service Oriented Architecture’ (SOA) and ‘Software as a Service’ (SaaS) are very popular, especially in enterprise contexts.

Unfortunately, due to the way services have been implemented and sold in the past, SOAs are associated with high costs and complexity, and deemed overkill for most web applications. However, services can be applied in more situations than one would think, and they can provide unexpected benefits.

In this article we discuss the so-called ‘service APIs’ – a term we use to promote a way of implementing web services without the negative associations of SOA. Service APIs constitute a lightweight, pragmatic approach to web services that is applicable in many situations.

The document starts by describing the typical problems that traditional web development causes, such as inconsistency, high maintenance costs, long time to market and vendor dependence. It then describes the concept of service APIs and how this concept can eliminate these problems. Finally, a plan of approach for implementing a service API is provided.

Service APIs

The traditional web development model

Each website in an organisation uses a separate database, and implements its own logic. Sometimes, websites share a common central database, but the websites are often still considered stand-alone.

This leads to a number of challenges that companies typically run into:

Inconsistency. If website 1 and website 2 both implement a feature for user registration, it is not a trivial task to ensure that both features work the same across both websites.
Cost of maintenance. If a feature needs to be changed, and the feature is implemented on multiple websites, the changes need to be made multiple times.
High per-website cost. In the event of a new website (e.g a website dedicated to a commercial event, such as the release of a movie or a product promotion), typically a vendor is approached that implements all functionality from top to bottom, not taking advantage of everything that has already been built for previous websites in the past.
Time to market. The above challenges have the side effect of increasing the time to market of new websites.
Vendor dependency. A common problem with top-to-bottom websites is that there is a large dependency on the vendor that implemented the website. It is difficult to attract new vendors to develop new functionalities, and there is a tight coupling between various parts of the system. E.g. in the case of a third party ticket system, its integration into the website is often very tight, meaning that the exchange of either the website or the third party system can be difficult.

Most websites start out with this traditional model, but as companies and their online presence grow, the above challenges at some point start to become prevalent.

The service API model

To overcome the disadvantages of the traditional approach, a better model for developing websites is required, one that is more suitable for growth and ongoing maintenance.

In this model the functionality that is common across websites is pushed down into a service layer. This includes the functionality that is typically part of core business.

The websites become smaller, in the sense that they contain less functionality. Instead of reinventing the wheel by implementing common functionality, they use standard functionality that is provided by the service API in a uniform way.

Typical characteristics of this architecture are:

Lower development cost. New functionality is implemented once and reused in many different websites (‘Build once, use anywhere’). Also, entirely new websites are less expensive, as they too can reuse functionality that the service API provides.
Less vendor dependent. It is important that you retain ownership of your service API. This ensures that you are in complete control of the functionality that the API provides, and as such, can instruct website vendors to make use of the functionality your API provides.
Faster time to market. Because of the availability of a common layer of core functionality, new websites or changes to websites can be developed in less time.

Multi-channel support

The service API model is also very suitable for multichannel approaches. The following picture depicts a scenario where various websites and applications are using the functionality provided by the API.

By making a part of the API public, it is possible to get third party developers to write applications based on the core functionality. This is a model that is typically used to encourage social network developers (e.g. Twitter, Facebook, LinkedIn) and independent website developers to build widgets and applications based on the core API.

In the case of an ecommerce system, this could mean that third party websites can easily integrate a point of sales for your store in their own website, effectively becoming resellers that directly use the core functionality of your business.

Integrating third-party applications

The service API model is very suitable for the integration of Third Party applications such as financial systems, data catalogs, CRM systems etc. The API can contain a service for every third party system that needs to be integrated.

Take the example of a theatre website selling tickets through their website by connecting their website to their ticketing system. The following diagram depicts how such a ticket system could be integrated into the service API.

This hypothetical example demonstrates a number of benefits that the service model offers:

Throttling. The capacity of a third party system is often not tailored to the demands of a high traffic internet site. Using a so-called throttle it is possible to overcome this problem. In the above scenario, a website requires twice the amount of capacity that the ticket system offers, and this is overcome by implementing a cache for the capacity and a queue for orders. For read-only data, a cache is implemented so that for a large number of website requests, only a single call to the backend system is necessary.
Distribution control. The ‘allotment’ box in the diagram symbolises a situation where the website owner has greater control of the use of his API. In the case of ticket sales, this could mean limiting the amount of tickets to a particular showing for a certain website. (E.g. discount website X is not allowed to book more than half of a venue.
Content enhancement. Sometimes more content is required than the third party provides. If adding such content to the third party system is not possible, the API can enhance the content with either manual effort (through a CMS that is connected to the service API) or using content providers such as Wikipedia, IMDB or commercial vendors of catalog data.

An important aspect in this approach is the sense of ownership. No longer are websites or vendors in control of what functionality is provided, but the owner of the service API is in control.

Example applications

Service APIs are applicable in a large number of situations. This section provides an illustration of typical environments that will benefit from a service API. Mind that these are only examples, as service APIs are applicable in a large number of situations. The illustrations in this section are intended to highlight how you could benefit from this architecture.

Media and broadcasting

In the broadcasting industry companies often manage a myriad of digital properties. In many situations, each individual programme or show has its own website, tailored to a specific target audience.

Time to market is essential; when a broadcaster purchases a particular show, it is important that before the show starts airing, the accompanying website is online. In the traditional approach, each individual website is a separate entity, as described earlier in this article. With the service model, a number of features can be built into a service:

Television schedule publication/syndication
Show rating
Commenting
Visitor profiles
Content managment
Custom analytics

These are just examples of functionality that can be ‘built once, used anywhere’. It even allows additional features such as ‘single sign on’, where fans of a certain television show can reuse their account on the websites of other shows they watch.

Publishing

In the publishing industry companies often deal with multiple publications (magazines, newspapers etc.). Since all of them typically have a different target audience it makes sense to have different websites. However, these websites can share a lot of common functionality:

Subscriptions
User profiles
Self service portal
Content syndication
Subscriber feedback

Often these sites even have the same content, for example information about the publisher, terms and conditions etc.

Again, a service layer is an ideal way to consolidate these common characteristics, and save money by not having to maintain each of them individually.

An additional aspect of service APIs might be interesting here: the ability to support mul- tiple channels. The same content and functionality can be reused not only on the website, but also in iPhone apps, feeds and, ultimately, print.

Travel

Travel is the industry that typically has the most ‘connected’ type of website. Connections to airline reservation systems, hotel booking backends, financial applications, insurance company systems, search portals; it’s hard to find a travel website without any external connections. This leads to a certain amount of complexity in the websites.

This complexity typically results in multiple interfaces. There’s an end-user interface for people that want to make holiday arrangements, there’s an operator interface that is use

by a customer support team, there’s an interface that provides reports to marketing departments and management, and often there are third party portals where agencies have their own specialised way of making reservations.

Finally, some companies service various target audiences and employ multiple brand names. All these characteristics lead to various systems or interfaces that need to be maintained, and a service API can help make sure that every wheel is only invented once. Functionality that can easily be moved into a service includes:

Availability of rooms
Creating reservations
Pricing
Customer Relationship Management (CRM)
Billing
Import and export of data
Connections to external systems

All applications, websites and third party systems can then use uniform API to access these features without duplicating business logic.

Implementing service APIs

In the previous sections, we discussed the concept of the service API. In this section, we will look at how the service API can be implemented. It’s not a matter of just creating an architecture and writing a bit of code. The service API is an important concept that requires a well thought-through process to optimise the benefits the service API offers.

Functional design

The implementation of a service API starts with an extensive functional design. In the functional design phase, two important aspects are investigated:

Required API functionality. The outcome will be a document describing the func- tionality that the API should offer to websites and applications, in as much detail as possible.
Required third party integration. If the API will need to connect to third party sys- tems, it is a good idea to investigate what the possibilities are to connect to the sys- tem, and what functionality needs to be changed/added by the third party supplier to make it possible to integrate the system in the API.

The goal of the functional design is to define which parts of the system will be exposed via the API. Equally important is to define which parts will remain private. After this phase, all the boundaries of the project should be clear to all parties involved.

Technical design

Once the functionality of the API has been agreed upon, a technical design should be created describing the technical details of the API. Where the functional design an- swers the ‘What?’ question, the technical design will answer the ‘How?’ question.

The following items are covered during this phase of the project:

Detailed API specification. A list of calls that websites can make to the API, and a list of calls that the API will perform on third party systems.
Protocol specification. The technical design will define what protocols and interchange formats (SOAP, REST, JSON, XML etc.) will be supported in the API.
Architecture. An overview of how the API will be built internally and how it will be able to meet both the functional and non-functional requirements.

Development

Once you have a clear picture of the API that you are going to build, it’s time to start development. PHP offers powerful functionality to create a service API. It has native support for creating and consuming web services of all kinds. Furthermore, there are many open source utilities and frameworks, such as the Zend Framework, that add an extra layer of productivity on top of PHP and make it even easier to implement a service API.

Since the most important preparations have already been done up front, it’s fairly straight- forward to implement the API according to the specifications from the Functional and Technical Design Phases.

Testing

Since the service API will play a vital role in potentially more than one system, it is important to perform functional tests and performance tests on the API. Functionality should be tested using unit tests (automated test scripts that ensure the continuous operation of a piece of software), or at least the business critical parts of the API.

Note that while listed as a separate phase in this article, if an Agile development methodology such as scrum is used to develop the API, testing will be an integral part of the development lifecycle and will be a continuous effort.

Documentation

While documentation should be an integral part of every development project, extra effort needs to be put in the documentation of a service API. Website developers (internal and/ or external) are going to have to use the API to make use of its functionality, and therefore need a level of documentation that will allow them to efficiently do that.

The result of this phase should be an API developer’s manual that third parties can use to efficiently develop websites or applications against the API.

Training

Most competent developers will be able to work with the API if they use the API developer’s manual that will be provided. Some developers however may have the need to receive training on the use of the API, especially if the API is large and multi-faceted. If such is the case, you may want to consider creating a training programme for those that have to work with the service API.

Migration

Once the service API has been developed, you may come to a point where you need to migrate existing websites or applications to use the new API.

Migration is completely optional. It is possible to use the API only for new developments while existing websites are not migrated. This will have the benefit of a reduced invest- ment, but the disadvantage of having legacy functionality that may not be consistent with the API and that may have a higher maintenance cost.

The amount of work depends on the level of integration and the current state of the appli- cations that need to be migrated.

An interesting feature of service APIs is that they allow a gradual, step-by-step migration path. It is possible to use an existing application almost out of the box with just a single API feature implemented in the existing code. Once this feature is migrated to the API successfully, another feature can be migrated. This way the migration is smooth and painless, and the API will gradually be tested against your existing real world applications.

Maintenance

Once the API has been built, it needs to be maintained. Typically maintenance consists of corrective maintenance (bug fixes etc.) and change management. Change manage- ment includes the addition of new features, changes in features, changes in the archi- tecture, and so forth; anything that the API requires throughout its lifetime.

By maintaining the API, websites can continue to maximise its value, and marketing and sales departments can continuously create new ideas and have the API support those ideas.

Conclusion

In this article, we have discussed an overview of the concept of service APIs, by describing the architecture, its benefits and the problems it solves. We’ve seen that traditional development architectures lead to a number of problems one organisations and their internet presence grow, and we’ve also seen that service APIs are a way to overcome all of them. Finally, we proposed a process for those interested in implementing a service API within their organisation.