Home / server / Questions / 1052294
Accepted
Jon Buys
Asked: 2021-02-04 08:49:25 +0800 CST

AWS Transfer SFTP Custom Identity Provider Private API Gateway

  • 772

I've setup AWS Transfer SFTP with CloudFormation and am using a custom Identity Provider setup with API Gateway fronting a Lambda function. Previously, my setup worked fine, but the API Gateway was public, and I wanted to make it private and bring it inside the VPC. I setup a VPC Interface Endpoint and associated it with the API Gateway. Relevant CloudFormation bits below:

  APIVPCEndpoint:
    Type: AWS::EC2::VPCEndpoint
    Properties:
        PrivateDnsEnabled: true
        ServiceName: com.amazonaws.us-east-1.execute-api
        VpcEndpointType: Interface
        SubnetIds:
            - subnet-11111111
            - subnet-22222222
        SecurityGroupIds:
            - sg-111111111111111
        VpcId: vpc-222222
   CustomIdentityProviderApi:
    Type: AWS::ApiGateway::RestApi
    Properties:
      Name: SFTP Custom ID Provider
      FailOnWarnings: true
      EndpointConfiguration:
        Types:
        - PRIVATE
        VpcEndpointIds:
        - Ref: APIVPCEndpoint

However, with this setup the DNS name for the API Gateway no longer resolves in DNS, and my SFTP instance can't reach it. I get an error:

{
    "Response": "",
    "StatusCode": 0,
    "Message": "Unable to call identity provider: Unable to execute HTTP request: randomname.execute-api.us-east-1.amazonaws.com: Name or service not known",
    "Url": "https://blablabla.execute-api.us-east-1.amazonaws.com/prod/servers/s-blablablabla/users/myusername/config"
}

I verified with dig and nslookup that the DNS is indeed not resolving. What does resolve is the name of the Endpoint, but, when I try to paste that name into the AWS Transfer Console as the Invocation URL for my custom identity provider, I get another error:

Failed to edit server details: Invalid API Gateway endpoint

I have a feeling that I've wandered into "unsupported configuration" territory, and for now I'm going to move the API Gateway back out of the VPC and make it public again so the system works. However, if anyone has done this and has any advice, I'd love to see if I could get the private configuration to work.

cloud amazon-web-services

2 Answers

  1. We just tried doing something similar (internet-facing SFTP server that is VPC hosted and using a private API Gateway + Lambda as a custom identity provider) and got direct confirmation from Amazon that the API Gateway endpoint currently has to be Regional in this scenario (for now, at least). We asked them to clarify this in their documentation and to add this to their roadmap; I'll try to update this response when they've updated the docs.

    • 3
  2. Best Answer

    Turns out the correct answer for our use case was not to have a Lambda calling the refresh cache API at all. In June 2020 AWS added a targeted automated cache refresh option to the File Gateway:

    This feature is based on the ‘duration since last access’ for each directory. All access requests that are made while the timer is still running treat the contents of the directory as current. After the timer expires, the next access of the directory results in a refresh of the directory. If the timer is 30 minutes, then the contents of that directory reflects the contents in Amazon S3 no longer than 30 minutes ago.

    The options range from 1 minute to several days. I've set ours to five minutes, and I'm betting that will be fine for us.

    Reference: https://aws.amazon.com/blogs/storage/automating-cache-refresh-process-for-file-gateway-on-aws-storage-gateway/

    • 0