78% of companies that implement microservices without an API gateway end up rewriting huge chunks of their architecture within 18 months. How crazy is that? And more importantly — why do so many of us still do that?
I guess the direct communication approach feels simpler for many. But only at first, until it turns into a headache so big, it makes your team cry. A solid microservice software architecture often needs smartly designed gateways. And in this read, we’ll dive into some API gateway use scenarios and how they can help your microservices.
The Microservices Communication Chaos
If you remember the “telephone” game many of us played as kids, you already have an idea of what microservices without a gateway are. One person whispers to another, and by the end, the message is totally garbled.
Let’s say, you have a system with 8 microservices and 3 client applications. It’s potentially 24 direct connections to maintain! Each of which represents:
- Duplicated security checks
- Inconsistent error handling
- Varying API styles and conventions
- Different authentication implementations
Netflix’s initial microservices architecture had hundreds of direct client-to-service connections. It means their tech team might have spent up to 40% of their time just managing these connections. The company addressed this issue through API gateways, circuit breakers, and service discovery mechanisms.
When every service handles its own authentication, it reinforces a system that is fundamentally vulnerable:
- A fintech startup I consulted for found out that three different teams had implemented three different JWT validation approaches. One of them had a critical vulnerability that stayed off the radar for months.
- Each service ends up with its own rate-limiting implementation. That makes a consistent defense against DDoS attacks barely possible.
- When you need to update authentication across your system, you have to engage with 20+ codebases instead of one.
So, managing microservices without API gateways is pretty much like herding cats. But the cats are on fire.
What’s an API Gateway?
An API gateway is a dedicated service, a middleman, between your clients and your microservices.
Otherwise speaking, it’s a single entry point that routes all API traffic to the appropriate microservices. Plus, an API gateway handles cross-cutting concerns such as authentication, logging, and rate limiting.
Its core responsibilities include:
- Authentication/authorization – Verifying who’s making the request and what they’re allowed to do.
- Request routing – Directing client requests to the correct service.
- Response transformation – Reformatting data from services to match client expectations.
- Rate limiting – Preventing any single client from overwhelming your services.
- Monitoring – Tracking all API traffic in one place.
Do not mix up API gateways and service meshes:
- Service meshes manage internal communication between your services (east-west traffic).
- API gateways manage external traffic coming into your system (north-south traffic).
Some companies (like Uber) use both: a gateway for clients and a mesh for service-to-service.
TOP 4 Gateway Benefits
#1 – More Efficient Monitoring
API Gateway automatically sends metrics such as error rates (4xx and 5xx), latency, and throughput to CloudWatch in near-real-time. This enables detailed monitoring and historical analysis over 15 months.
Centralized gateway monitoring gives you:
- End-to-end tracing across services;
- Consistent logging format across all APIs;
- Complete visibility of all external requests;
- Real-time alerting on traffic anomalies.
#2 – Better Security
Consolidating authentication at the API gateway can reduce security-related code by 84% across microservices.
With a gateway:
- Apply consistent rate limiting to prevent denial of service attacks;
- Enforce IP whitelisting/blacklisting in one place;
- Implement JWT validation, API keys, or OAuth2 flows once;
- Add new security measures instantly across all services.
#3 – Legacy Support Without Service Rewrites
If you are dealing with a legacy system, the gateway’s transformation capabilities can be employed instead of rewriting old services.
In this regard, your gateway can:
- Handle pagination differences between old and new clients;
- Provide backward compatibility for older API versions;
- Transform requests from legacy XML formats to the JSON your new services expect;
- Aggregate results from multiple services into a single response.
#4 – Traffic Control That Prevents Disasters
Suppose an upstream service is experiencing repeated failures (e.g., HTTP status codes 500 or 503). If there’s no circuit breaker, the API gateway will continue retrying requests. This can potentially cause cascading failures across the system.
However, this issue can be solved with the Apache APISIX API Breaker Plugin which monitors consecutive failures. If the number of failures exceeds a defined threshold, the circuit breaker immediately returns exceptions for new requests (instead of forwarding them to the failing service).
In this scenario, effective gateways provide:
- Circuit breaking that prevents overwhelmed services from failing completely;
- Load balancing that distributes traffic evenly;
- Timeouts that prevent one slow service from creating a backlog;
- Request retries that handle transient failures automatically.
Ultimate Gateways Comparison
Different gateways excel in different environments:
| Gateway | Best For | Notable Users | Performance |
| Kong | Multi-cloud deployments | Nasdaq, SkyScanner | Up to 50,000 req/s per node. |
| AWS API Gateway | AWS-native architectures | Capital One, Snap | Infinitely scalable with higher latency. |
| Apigee | Enterprise with complex governance | Walgreens, Pitney Bowes | High throughput with 30ms typical overhead. |
| Tyk | Open-source with commercial support | Starbucks, GlaxoSmithKline | 10,000+ req/s on modest hardware |
But What About…? Addressing Common Concerns
“We Can Just Add It Later When We Need It”
This is a costly misconception. Trying to fit a gateway into an existing architecture usually ends up costing three to five times more than if you had built it in from the beginning.
Here’s why:
- You’ll find hidden direct dependencies between services.
- All client applications will need updates.
- You might run into incompatible API design patterns.
- Consistent authentication across all services will need to be implemented.
“Won’t It Add Unacceptable Latency?”
The numbers tell a different story:
- Kong typically adds about 3-5ms of latency in most setups.
- AWS API Gateway averages around 29ms.
- Custom-built gateways can be optimized for specific needs, often getting under 1ms.
When you compare this to the average API call latency of 200-500ms, the overhead from the gateway is pretty minimal. Plus, don’t forget the advantages of consistent caching and connection pooling that can really boost overall performance.
“Won’t the Gateway Become a Single Point of Failure?”
This concern stems from a misunderstanding of how to properly deploy a gateway. If a gateway goes down, traffic is automatically rerouted to other available checkpoints, so users won’t experience any downtime.
Modern API gateways are designed to be deployed in clusters, which means that a well-implemented gateway indeed enhances system resilience by centralizing failure management.
Your Gateway Implementation Roadmap
Phase 1: Foundation (Weeks 1-2)
- Select gateway technology based on your environment and requirements.
- Deploy a minimum of two gateway instances with load balancing.
- Implement basic request logging and monitoring.
- Route a single, low-risk service through the gateway.
Phase 2: Core Policies (Weeks 3-4)
- Implement an authentication mechanism (JWT, API keys, or OAuth2).
- Add basic rate limiting for all endpoints.
- Create a staging environment gateway for testing.
- Set up alerting for gateway health and performance.
Phase 3: Expansion (Weeks 5-8)
- Migrate additional services behind the gateway.
- Implement service-specific rate-limiting policies.
- Add request validation for common parameters.
- Configure detailed analytics and monitoring dashboards.
Phase 4: Optimization (Ongoing)
- Implement response caching where necessary.
- Define different SLAs for different API consumers.
- Set up automated gateway performance testing.
- Create governance processes for gateway configuration changes.
Migration tip: Use the strangler pattern: gradually route more traffic through your gateway instead of attempting a “big bang” cutover. And if you’re not sure what is best for your microservices, consider microservice consulting provided by companies like Velvetech.
