AWS API Gateway with AppRunner
AWS AppRunner is a reasonably new service that aims to simplify deploying and managing web based applications and REST endpoints.
New, in the sense that the AWS CDK does not offer stable L2 constructs yet for the service.
New also, in the sense that some capabilities available on the service are under-documented and difficult, if not impossible, to enable via the CDK.
Custom domains over AppRunner are once such capability.
Another such capability is having the AWS API Gateway front a given private AppRunner instance.
Without a long winded overview and explanation, if you find yourself with an AppRunner instance on a private VPC that is intentionally not publicly accessible, it is possible to place an API Gateway in front of one or more AppRunner instances.
First, making an AppRunner instance not publicly accessible requires dropping down to the L1 constructs.
Service service = ...;
// in order to make an app runner service private, we need use L1 constructs to set the network configuration
CfnService.NetworkConfigurationProperty networkConfiguration = CfnService.NetworkConfigurationProperty.builder()
.ingressConfiguration(CfnService.IngressConfigurationProperty.builder()
.isPubliclyAccessible(application.publiclyAccessible()) // this api is not on the L2 construct
.build())
.egressConfiguration(CfnService.EgressConfigurationProperty.builder()
.egressType("VPC")
.vpcConnectorArn(vpcConnector.getVpcConnectorArn())
.build())
.build();
CfnService cfnService = (CfnService) service.getNode().getDefaultChild();
Objects.requireNonNull(cfnService).setNetworkConfiguration(networkConfiguration);
When this happens, the AppRunner no longer allocates a domain name for access within the VPC. This requires an InterfaceVpcEndpoint.
Service service = ...;
String region = ...;
Vpc vpc = ...;
InterfaceVpcEndpoint endpoint = InterfaceVpcEndpoint.Builder.create(service, "InterfaceEndpoint")
.vpc(vpc)
.subnets(subnetSelection)
.securityGroups(securityGroups)
.service(new InterfaceVpcEndpointService("com.amazonaws.%s.apprunner.requests".formatted(region)))
.privateDnsEnabled(false)
.open(true)
.build();
CfnVpcIngressConnection ingressConnection = CfnVpcIngressConnection.Builder.create(service, "EndpointConnection")
.vpcIngressConnectionName(name.with("IngressConnection").lowerHyphen())
.ingressVpcConfiguration(CfnVpcIngressConnection.IngressVpcConfigurationProperty.builder()
.vpcId(vpc.getVpcId())
.vpcEndpointId(endpoint.getVpcEndpointId())
.build())
.serviceArn(service.getServiceArn())
.build();
You can get the domain name from ingressConnection.getAttrDomainName(). This will be required below.
Next up, we have to put a Network Load Balancer listener in front of the InterfaceVpcEndpoint, to do that we are required to make a target group of the endpoint IPs. Sadly, we can only ask the endpoint for endpoint.getVpcEndpointNetworkInterfaceIds(), ENI ids, and even those have to be lazily resolved.
This is where, the new to me construct, AwsCustomResource comes in.
Service service = ...;
String region = ...;
Vpc vpc = ...;
NetworkLoadBalancer balancer = NetworkLoadBalancer.Builder.create(this, "NetworkLoadBalancer")
.loadBalancerName("nlb")
.vpc(vpc)
.internetFacing(false)
.crossZoneEnabled(true)
.vpcSubnets(subnetSelection)
.securityGroups(securityGroups)
.ipAddressType(IpAddressType.IPV4)
.build();
// look up ips for the enis
// we cannot ask the endpoint for the number of enis via getVpcEndpointNetworkInterfaceIds() as it returns a single array token
// not representative of the actual number of enis
// we assume one ip per subnet given to the endpoint
List<String> outputPaths = IntStream.range(0, subnetSelection.getSubnets().size())
.mapToObj("NetworkInterfaces.%s.PrivateIpAddress"::formatted)
.toList();
// we need to use a custom resource to get the private ip addresses of the network interfaces
AwsCustomResource customResource = AwsCustomResource.Builder.create(this, "EniLookup")
.functionName(physicalName)
/// do not do this. will cause the stack to hang on deployment, and prevents an easy deletion
// .vpc(vpc).vpcSubnets(subnetSelection)
// actually installs during the event, let's rely on the installed version
.installLatestAwsSdk(false)
.timeout(Duration.minutes(5))
// by default onCreate calls onUpdate, so we only need to define onUpdate
.onUpdate(AwsSdkCall.builder()
.physicalResourceId(PhysicalResourceId.of(physicalName))
.service("EC2")
.action("describeNetworkInterfaces")
.parameters(OrderedSafeMaps.of(
"NetworkInterfaceIds", endpoint.getVpcEndpointNetworkInterfaceIds()
))
.outputPaths(outputPaths)
.build())
.policy(AwsCustomResourcePolicy.fromStatements(List.of(
PolicyStatement.Builder.create()
.actions(List.of("ec2:DescribeNetworkInterfaces"))
.resources(List.of("*"))
.build()
)
)
)
.build();
List<IpTarget> ipTargets = outputPaths.stream()
.map(customResource::getResponseField)
.map(IpTarget::new)
.toList();
NetworkTargetGroup targetGroup = NetworkTargetGroup.Builder.create(this, "TargetGroup")
.targetGroupName("app-runner")
.port(443) // the port apprunner presents
.protocol(Protocol.TCP)
.vpc(vpc)
.targetType(TargetType.IP)
.targets(ipTargets)
.build();
NetworkListener networkListener = balancer.addListener(name.with("Listener").camelCase(), NetworkListenerProps.builder()
.protocol(Protocol.TCP)
.port(1001) // arbitrary, but don't duplicate across multiple listeners on this balancer
.defaultTargetGroups(List.of(targetGroup))
.loadBalancer(balancer)
.build());
You will need the listener ARN, networkListener.getListenerArn(), in a later step.
Now we create the API Gateway V2.
HttpApi httpApi = HttpApi.Builder.create(this, "HttpApi")
.apiName("api")
.createDefaultStage(true)
.disableExecuteApiEndpoint(true) // turns off the default endpoint, optional
.build();
You may find it useful to customize the logging.
LogGroup logGroup = LogGroup.Builder.create(this, Label.of("LogGroup").with(id).camelCase())
.logGroupName("/aws/apigateway/" + name.lowerHyphen())
.removalPolicy(RemovalPolicy.DESTROY)
.retention(RetentionDays.FIVE_DAYS)
.build();
((CfnStage) httpApi.getDefaultStage().getNode().getDefaultChild()).setAccessLogSettings(CfnStage.AccessLogSettingsProperty.builder()
.destinationArn(logGroup.getLogGroupArn()
.format("$context.identity.sourceIp - - [$context.requestTime] \"$context.httpMethod $context.routeKey $context.protocol\" $context.status $context.responseLength $context.requestId")
.build());
In this last step, we integrate the API Gateway to the balancer listener via a VpcLink over a RESTful route. Obvious, right?
String host = ...; // the domain name from the ingressConnection
String listenerArn = ...; // from the listener
// if VpcLink.Builder.create is used instead, api gateway returns 503
VpcLink link = httpApi.addVpcLink(VpcLinkProps.builder()
.vpcLinkName("vpc-link")
.vpc(vpc) // where the apprunner service resides
.subnets(subnets)
.securityGroups(securityGroups)
.build());
NetworkListener listener = NetworkListener.fromNetworkListenerArn(this, serviceName.camelCase(), listenerArn)
HttpNlbIntegration integration = HttpNlbIntegration.Builder.create("app-runner", listener)
.method(HttpMethod.ANY)
.parameterMapping(new ParameterMapping()
.overwritePath(MappingValue.custom("/${request.path.proxy}")) // see path below
.overwriteHeader("Host", MappingValue.custom(host)) // since this is over tls, need to set the host header
)
.vpcLink(vpcLink)
.secureServerName(host) // since this is over tls, we need to set the server name
.build();
String path = "%s{proxy+}".formatted("/api/"); // translate /api/.... to /....
AddRoutesOptions apiRoute = AddRoutesOptions.builder()
.path(path)
.methods(List.of(HttpMethod.ANY))
.integration(integration)
.build();
Without setting secureServerName(), a 503 error will be returned.
Without setting .overwriteHeader("Host", MappingValue.custom(host)) a 404 error will be returned. This particular issue is quite maddening since the back-end hosted service isn’t returning the 404, it’s returned from AppRunner (envoy) because it doesn’t recognize the host header passed by the client to the API Gateway.
Of note, by testing API endpoints with curl -sD - you can see the returned headers. If the server header has the value envoy, I’m going to say you can assume AppRunner is in the request path. envoy also is returned for 200 responses. It would be wise to replace this with the public facing domain name.
Network security issues are harder to sort out. For those you’ll see a 503 response, and if you check the logs in CloudWatch, look for the $context.error.responseType value of INTEGRATION_NETWORK_FAILURE. You’ll need to add this variable above to the logging format – see this list of other useful logging variables.
For simplicity I left out a number of constructs and configuration, but it’s reasonable to drop everything into the same VPC, subnets, and security group, in order to get things working.
Also note I also have some clusterless-commons API calls littered about in the above code.
Github issues of note:
