The emerging API Economy has led to organisations having to become more agile and reactive than they’ve ever been before to stay competitive. In the old days, you might have had the luxury of 6-12 months to take an idea through to delivery – today most companies would expect delivery in 1-3 months!
We still have all of the enterprise data and systems to integrate, but now we've also got APIs and potentially rapidly changing mobile apps to think about.
Building new distributed systems is understandably complex, but who owns that complexity, and how can they be smarter about getting solutions out of the door quicker, despite it?
The new complexity
Consider the ‘Old World’ way of working for a moment - when SOA was king - APIs have provided us with much simpler, basic standards. We’ve moved away from the plethora of Web Services (WS) standards and focused on standards such as HTTP, REST and JSON. We've designed simpler data models that are self-describing and focused on providing what the consumer requires, not what the provider needs.
So it seems that we've managed to reduce complexity considerably in the 'New World' - where 'apps' now rule the roost?
Well, we have to add in new standards like OAuth and OpenID Connect to replace some of the WS standards we've lost. We also have to consider social integration, developer portals, API monetisation and scalability - things we rarely had to worry about doing B2B integration.
Security also becomes a far greater concern that it ever was before - via APIs, you are being expected to open up your organisation to the internet, full of rogue apps, hackers, and other nasties.
So it’s still very complicated stuff.
You're thinking "I thought that APIs were meant to make things quicker?"
Maybe you're also thinking "How am I supposed to do this in one month if it's just as complex and it was taking me six months before?"
Let’s investigate this complexity further.
SOA: No-one gets it easy
With SOA, you might choose five or six out of the hundreds of WS standards to implement. Your service consumer (business partner usually) would also need to learn those in detail to consume your service. When that business later had to integrate with another business partner, the chances are they'd chosen five or six different ones, so they have to learn those too.
Also, services tended to be exposed with complex data formats, sometimes representing some sort of common or canonical enterprise format that you had designed. The consumer was forced to understand this too, so that they could create and consume the messages.
In your organisation, once you had understood those standards in their entirety, you often needed to develop full stacks of software to process them.
And of course 50% if the time, your company was in the role of consumer for these B2B integrations - and had the other company’s complexity to get to grips with. But integration with the outside world was mainly business to business (B2B) - you did these two or three times a year or so, and lived with it.
What we seen in Figure 1 is an equal sharing of the inherent complexity of doing integration. No-one gets it easy.
Figure 1. Percentage split of complexity responsibility in SOA.
The consumer is king
In the API World, the consumer is king and simplicity of the interface is paramount.
We still have lots of complexity, such as supporting new standards like OAuth as we mentioned earlier, but now the provider (you) has to handle as much of that complexity as possible to make the consumer’s job easier – shifting the ownership of the complexity heavily towards the business exposing these APIs.
In OAuth for example, the consumer need only call a simple RESTful API, and then ensure the token returned is used to call protected APIs. It's very easy for that consumer - and this time they might not be a business partner, they could be one of 100s of developers out on the internet writing mobile apps to use your APIs.
But the provider of the API gets the short straw here - they are responsible for implementing the non-trivial complexity of the OAuth 2.0 server standard – storing client ID, tokens, validating, expiring, scope checking, etc.
You see a similar uneven ownership split wherever complexity appears with APIs involved:
Figure 2. Percentage split of complexity responsibility with APIs.
And unlike those two or three big B2B integrations that you used to do per year, you're now expected to be doing this sort of integration as business-as-usual, continuously, and at an order of magnitude quicker than you were before.
As an organisation in the 'provider' role - this poses some major challenges for you:
- you need to get 5-10 times quicker at getting solutions out the door
- your IT team needs to implement considerable complexity - most of it new to them
- it needs to be more robust than ever before – security is paramount now you’re exposing APIs on the internet
API Management and API Gateways
Firstly, the solution for all this complexity you now own is - buy it don't build it.
A market leading and mature API Management platform (including an API Gateway) will give you it all 'for free'. Gartner make the same point in their interesting '10 steps to the API Economy' article.
Not only does this save a massive amount of time, it creates consistency and adds a layer of robust security around everything you expose: An API Gateway is your gatekeeper, protecting your enterprise assets from threats.
Buying a tried and tested, secure solution gives you a platform that you can be assured is security hardened, regularly penetration tested and kept up to date and patched against emerging vulnerabilities.
And as solutions around secure access to APIs get more complicated (e.g. including OAuth2.0, open ID connect and two factory authentication mechanisms), it's important that your platform handles that complexity, freeing your IT department up to concentrate on building functionality for business benefit and not on the 'infrastructure'.
Getting all this complexity 'for free' by buying a market leading solution is great, but it's of limited use if it still takes you months and months to build the APIs themselves.
If it takes your IT department six months to build a solution around SOA business services today, adding an API layer on top is clearly only going to make it take even longer.
API Creation is becoming a new core competency of your IT department. Building APIs traditionally using extensive coding and middleware is not going to cut it anymore if you want to start seeing those orders of magnitude speed increases. It's slow, error prone and can become very complex.
Gartner coined the term, bimodal IT as a possible solution:
"Bimodal IT is the practice of managing two separate, coherent modes of IT delivery, one focused on stability and the other on agility. Mode 1 is traditional and sequential, emphasizing safety and accuracy. Mode 2 is exploratory and nonlinear, emphasizing agility and speed."
Embracing a Mode 2 strategy within your organisation, which facilitates rapid API creation, will be key to meeting the API Economy challenge.
Just as in the case of 'build not buy' for the implementation of complex standards like OAuth, think 'config not code' for building APIs.
Smart Products which allow enterprise quality, complex APIs to be built in days not months are required if you are to meet your agility goals. Traditional coding and tooling will no longer cut it.
A full featured API Gateway (such as CA's API Gateway) can step into this role - enabling you to rapidly build APIs, integrating with existing resources within your organisation such as SOA services or even databases directly. And, in the case of CA's API Gateway, with the simplicity of a drag and drop GUI interface.
Going forward, you'll probably want to extend your API Creation capability - and it's here we are seeing the emergency of an entirely new product segment around rapid API creation. CA is pioneering in this segment with their Live API Creator product - allowing you to leverage the value of the data in your systems of record data sources, exposing it as first class APIs or microservices in a matter of hours.
If this article has raised any questions or you’d be interested in discussing any of the specifics mentioned, please don’t hesitate to drop me a line.