In my last post I presented a theory about how digital transformation cannot be willed into existence, but requires a purposeful building of new habits. I’d like to use this as a foundational theme for this and future posts, exploring the sorts of habits we need to develop in order succeed, and outline how we might go about it.
APIs are driving transformation
The theme of transformation is a pertinent one. I don’t think I’m alone in believing that today’s technologies represent an inflection point in history, where advances seem to come as significant leaps of possibilities, rather than just incremental improvements in performance. This is affecting every aspect of our lives and is probably why the term ‘digital’ is being slapped onto everything; from homes, to job titles, to experiences!
These rapid changes are underpinned by the concept of sharing functionality through APIs. The open, sharing approach that APIs support has allowed disparate ideas from across the globe to be combined in new and unintended ways, basically mass experimentation, from which serendipitous leaps in innovation drop out.
Trust is integral to APIs
Now the temptation here might be to delve into the technical details of how an API is constructed, or perhaps the architectural patterns for an API ecosystem. For today however, I’d like to get to the real nub of the API challenge, and I think it all boils down to the fundamental concept of trust.
And we can view trust from three perspectives:
- Construction: The trust required between the stakeholders and participants involved in the design and build of API ecosystems.
- Identity: The trust needed by the API to translate the caller into a real world person or entity, with a proven identity.
- Access: The technical aspect of trust required to control access to APIs.
Before you embark on your API implementation, you need to have a plan for how you deal with all these perspectives on trust. In this post we’ll cover the construction angle because this will be the foundation of your API programme.
Can’t manage trust out of the system
Now past behaviours would suggest that we try to eliminate the need for trust by putting in a strict series of checks and controls to make sure everyone delivers to a tight set of requirements.
This approach barely works for a single, well understood, self-contained system. With an API ecosystem you’re building and evolving a lot of independent components, and the risk is we’ll quickly end up in a slow and costly management quagmire.
Instead, we need to take a balanced and considered approach.
Using inherent trust is efficient
In an ideal world, and perhaps for startups and greenfield projects, we would have a single, multi-disciplined team per API. Each team would have a domain expert/architect, a couple of API designers, a small implementation team of developers (dev-ops) and testers.
Each team will be autonomous and accountable for API implementation, from requirements to deployment. This removes the necessity for repeated handovers between specialist teams, or put another way, capitalises on the inherent trust and knowledge built up within a contained and familiar team. This approach is efficient, effective, and in theory, scalable.
This approach allows the number of teams to be spun up or down as needed in order to manage your API pipeline. It’s analogous to the horizontal scaling model in the cloud world, where you just add/remove machines to an environment to meet demand.
Figure 1. Horizontal scaling of dedicated API teams is normally limited by availability of domain experts and designers.
Trusted API design teams are more realistic
Unfortunately the real (brownfield) world is a little more complicated.
For a start, domain experts (architects) are usually a rare and expensive resource. There are only a few to go around and they need to be spread thinly across projects and other commitments.
Also, implementation teams have largely become commoditised and are usually outsourced, often nearshore or offshore.
The combination of these factors instantly breaks our nice horizontal scaling model. So we have to find a way of getting the most of out of a limited supply of domain experts, while keeping our global development teams properly utilised.
This is where the API Design Office comes in. It consists of a focused team of designers and business analysts who are charged with capturing the tacit knowledge from the domain experts and translating this into implementable designs for the development teams.
The intention is to mitigate the bottleneck caused by the limited supply of domain experts, ensuring that they are focused on where they can add the most value. Additionally, they can provide consistent and appropriate governance and knowledge capture activities, enabling continual improvement in the whole process.
Referencing our cloud world analogy again, this is more like a vertical scaling model, where you add/remove resources in a given team to meet demand, as opposed to adding more teams.
Formalise trust between the layers
We can’t ignore the other end of this approach where we have introduced an interface between the Design Office and the external development teams, where there isn’t necessarily an inherent level of trust.
This situation needs a more formal way to capture and communicate the API design. The most resilient solution is to agree on a domain specific language (or specification language), preferably using tools such as swagger.io or apiary.io. The key benefit to using a formal specification and framework is that the designs can be testable, which gives us assurances (i.e. trust), around quality and coverage.
Figure 2. API Design Office mitigates the limited supply of domain experts and manages the trust gap with testable specifications.
Don’t forget the big picture
Once we breakup the process across teams, we need to watch out for focusing on the performance of each layer, and forget that we actually care about the throughput of APIs being implemented.
This can easily result in a breakdown of stakeholder trust in the wider programme, with the temptation to ramp up controls and/or resources at the different layers to try and increase the rate of delivery, but usually only exacerbating the underlying problem.
We can address this in two ways:
1) Hold all the different participants and stakeholders accountable to a given delivery rate of APIs from the process. This encourages everyone to focus on efficient throughput, and not just delivering their particular activity. This can be as simple as a KPI metric based on the final tangible outcomes, thus reinforcing the message that all the teams need to collaborate.
2) Ensure that the whole process is completely transparent to all the stakeholders. Tasks and artefacts should be captured into a workflow tool so that all involved can see and comment on what’s going on, with automated dashboards acting as summary reports.
We cannot underestimate the impact of the second point. It’s definitely scary at first and the teams will undoubtedly feel vulnerable when failures as well as successes are both on show. However, it will quickly become apparent there is an enormous amount of effort to be saved by not having to constantly present a positive message to stakeholders, and vice versa where stakeholders are not look for new angles in order to prize out the truth of the situation.
Atlassian’s Jira is a capable and inexpensive workflow tool, and arguably has become a defacto standard amongst the development community.
This first article in the series has outlined why the concept of trust is integral to designing an efficient and effective API development programme. Because regardless of the technical excellence of your API components, if you can’t deliver them efficiently enough, to a high level of quality, then your users will quickly find an alternative.
Fundamentally we’re just trying to focus our API teams on the desired outcome. In our experience, everyone in the supply chain wants to do a good job, but we often let over-engineered processes prevent effective collaboration.
Trust is a key tenet in encouraging supportive behaviours and forming new habits. And ultimately it’s these new habits that will lead to transformation.
The next post in the series will tackle another angle on trust: how we build confidence in the true identity of an API user.