Fix imports for localstack_cli standalone package; re-enable CI for aws-proxy

.github/workflows/aws-proxy.yml

@@ -1,17 +1,15 @@
 name: LocalStack AWS Proxy Extension Tests
 
 on:
-# TODO: temporarily disabled - AWS proxy codebase needs to be fixed, to accommodate recent
-# changes in CLI and runtime repos, see: https://github.com/localstack/localstack/pull/13704
-#  push:
-#    paths:
-#      - aws-proxy/**
-#    branches:
-#      - main
-#  pull_request:
-#    paths:
-#      - .github/workflows/aws-proxy.yml
-#      - aws-proxy/**
+  push:
+    paths:
+      - aws-proxy/**
+    branches:
+      - main
+  pull_request:
+    paths:
+      - .github/workflows/aws-proxy.yml
+      - aws-proxy/**
   workflow_dispatch:
 
 jobs:
@@ -57,7 +55,7 @@ jobs:
           docker pull public.ecr.aws/lambda/python:3.8 &
 
           # install latest CLI packages
-          pip install --upgrade localstack localstack-ext
+          pip install --upgrade localstack
 
           # install dependencies
           sudo apt-get update

aws-proxy/README.md

@@ -21,52 +21,55 @@ The AWS Cloud Proxy can be used to forward certain API calls in LocalStack to re
 
 **Warning:** Be careful when using the proxy - make sure to _never_ give access to production accounts or any critical/sensitive data!
 
-**Note:** The Cloud Proxy CLI currently works only when installing the `localstack` CLI via `pip`.
-If you're downloading the `localstack` CLI as a [binary release](https://docs.localstack.cloud/getting-started/installation/#localstack-cli), then please use the proxy configuration UI described below.
-
 ### Usage
 
-#### CLI
-For example, in order to forward all API calls for DynamoDB/S3/Cognito to real AWS, the proxy can be started via the CLI as follows:
+#### Using curl (API)
 
-1. Start LocalStack via CLI
-```
-$ localstack start -d
-```
-2. Enable LocalStack AWS Proxy from the Web Application Extension Library
-3. After installation restart Localstack
-4. Install the AWS Proxy CLI package
-```
-$ pip install localstack-extension-aws-proxy
-```
-5. Configure real cloud account credentials in a new terminal session to allow access
+The proxy can be enabled and disabled via the LocalStack internal API. This is the recommended approach.
+
+1. Start LocalStack and install the AWS Proxy extension (restart LocalStack after installation).
+
+2. Enable the proxy for specific services (e.g., DynamoDB, S3, Cognito) by posting a configuration along with your AWS credentials:
 ```
-$ export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...
+$ curl -X POST http://localhost:4566/_localstack/aws/proxies \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "config": {
+      "services": {
+        "dynamodb": {},
+        "s3": {},
+        "cognito-idp": {}
+      }
+    },
+    "env_vars": {
+      "AWS_ACCESS_KEY_ID": "<your-access-key-id>",
+      "AWS_SECRET_ACCESS_KEY": "<your-secret-access-key>",
+      "AWS_SESSION_TOKEN": "<your-session-token>"
+    }
+  }'
 ```
-6. Start proxy in aforementioned terminal session via the CLI
+
+3. Check the proxy status:
 ```
-$ localstack aws proxy -s dynamodb,s3,cognito-idp
+$ curl http://localhost:4566/_localstack/aws/proxies/status
 ```
-7. Now, when issuing an API call against LocalStack (e.g., via `awslocal`), the invocation gets forwarded to real AWS and should return data from your real cloud resources.
 
-#### Proxy Configuration UI
-
-1. Start Localstack with extra CORS
+4. Disable the proxy:
 ```
-EXTRA_CORS_ALLOWED_ORIGINS=https://aws-proxy.localhost.localstack.cloud:4566 localstack start -d
+$ curl -X POST http://localhost:4566/_localstack/aws/proxies/status \
+  -H 'Content-Type: application/json' \
+  -d '{"status": "disabled"}'
 ```
 
-2. Enable Localstack AWS Proxy from the Web Application Extension Library
-
-3. Once the extension is installed, it will expose a small configuration endpoint in your LocalStack container under the following endpoint: http://localhost:4566/_localstack/aws-proxy/index.html . 
+5. Now, when issuing an API call against LocalStack (e.g., via `awslocal`), the invocation gets forwarded to real AWS and should return data from your real cloud resources.
 
-4. Use this Web UI to define the proxy configuration (in YAML syntax), as well as the AWS credentials (AWS access key ID, secret access key, and optionally session token) and save configuration. The proxy should report enabled state and on the host a proxy container should spawn.
+#### Using the LocalStack Web App
 
-![configuration settings](etc/proxy-settings.png)
+You can also configure the proxy from the LocalStack Web App at https://app.localstack.cloud. Navigate to your instance and use the AWS Proxy extension settings to enable/disable the proxy and manage credentials.
 
-5. Now we can communicate with the real AWS cloud resources, directly via LocalStack.
+Alternatively, the extension exposes a local configuration UI at http://localhost:4566/_localstack/aws-proxy/index.html (requires starting LocalStack with `EXTRA_CORS_ALLOWED_ORIGINS=https://aws-proxy.localhost.localstack.cloud:4566`). Use this Web UI to define the proxy configuration (in YAML syntax) and AWS credentials, then save the configuration. To clean up the running proxy container, click "disable" in the UI.
 
-To clean up the running proxy container simply click "disable" on the Cloud Proxy UI.
+![configuration settings](etc/proxy-settings.png)
 
 ### Resource-specific proxying
 
@@ -90,10 +93,7 @@ services:
     execute: false
 ```
 
-Store the configuration above to a file named `proxy_config.yml`, then we can start up the proxy via:
-```
-localstack aws proxy -c proxy_config.yml
-```
+Pass this configuration in the `config` field of the `POST /_localstack/aws/proxies` request body (as shown above).
 
 If we then perform local operations against the S3 bucket `my-s3-bucket`, the proxy will forward the request and will return the results from real AWS:
 ```
@@ -118,8 +118,6 @@ In addition to the proxy services configuration shown above, the following confi
 * `PROXY_LOCALSTACK_HOST`: the target host to use when the proxy container connects to the LocalStack main container (automatically determined by default)
 * `PROXY_DOCKER_FLAGS`: additional flags that should be passed when creating the proxy Docker containers
 
-**Note:** Due to some recent changes in the core framework, make sure to start up your LocalStack container with the `GATEWAY_SERVER=hypercorn` configuration enabled, for backwards compatibility. This will be fixed in an upcoming release.
-
 ## Resource Replicator CLI (deprecated)
 
 Note: Previous versions of this extension also offered a "replicate" mode to copy/clone (rather than proxy) resources from an AWS account into the local instance.
@@ -129,6 +127,7 @@ If you wish to access the deprecated instructions, they can be found [here](http
 
 ## Change Log
 
+* `0.2.4`: Replace deprecated `localstack aws proxy` CLI command with direct Python/HTTP-based proxy startup; update README with curl-based usage instructions
 * `0.2.3`: Enhance proxy support and tests for several services (API Gateway v1/v2, CloudWatch, AppSync, Kinesis, KMS, SNS, Cognito-IDP)
 * `0.2.2`: Refactor UI to use WebAppExtension pattern
 * `0.2.1`: Restructure project to use pyproject.toml

aws-proxy/aws_proxy/client/auth_proxy.py

@@ -3,8 +3,6 @@
 import logging
 import os
 import re
-import subprocess
-import sys
 from functools import cache
 from typing import Dict, Optional, Tuple
 from urllib.parse import urlparse, urlunparse
@@ -24,10 +22,6 @@
     LOCALHOST_HOSTNAME,
 )
 from localstack.http import Request
-from localstack.pro.core.bootstrap.licensingv2 import (
-    ENV_LOCALSTACK_API_KEY,
-    ENV_LOCALSTACK_AUTH_TOKEN,
-)
 from localstack.utils.aws.aws_responses import requests_response
 from localstack.utils.bootstrap import setup_logging
 from localstack.utils.collections import select_attributes
@@ -50,6 +44,7 @@
 from aws_proxy.shared.models import AddProxyRequest, ProxyConfig
 
 LOG = logging.getLogger(__name__)
+
 LOG.setLevel(logging.INFO)
 if localstack_config.DEBUG:
     LOG.setLevel(logging.DEBUG)
@@ -75,16 +70,33 @@ def __init__(self, config: ProxyConfig, port: int = None):
         super().__init__(port=port)
 
     def do_run(self):
+        self.register_in_instance()
+        self.run_server()
+
+    def run_server(self):
         # note: keep import here, to avoid runtime errors
         from .http2_server import run_server
 
-        self.register_in_instance()
         bind_host = self.config.get("bind_host") or DEFAULT_BIND_HOST
         proxy = run_server(
             port=self.port, bind_addresses=[bind_host], handler=self.proxy_request
         )
         proxy.join()
 
+    @classmethod
+    def start_from_config_file(cls, config_file: str, port: int):
+        config = json.load(open(config_file))
+        config["bind_host"] = "0.0.0.0"
+        cls(config, port=port).run_server()
+
+    @staticmethod
+    def register_proxy(
+        config_file: str, port: int, localstack_host: str, localstack_port: int = 4566
+    ):
+        config = json.load(open(config_file))
+        url = f"http://{localstack_host}:{localstack_port}{HANDLER_PATH_PROXIES}"
+        requests.post(url, json={"port": port, "config": config})
+
     def proxy_request(self, request: Request, data: bytes) -> Response:
         parsed = self._extract_region_and_service(request.headers)
         if not parsed:
@@ -464,8 +476,7 @@ def start_aws_auth_proxy_in_container(
         "AWS_ACCESS_KEY_ID",
         "AWS_SESSION_TOKEN",
         "AWS_DEFAULT_REGION",
-        ENV_LOCALSTACK_API_KEY,
-        ENV_LOCALSTACK_AUTH_TOKEN,
+        "LOCALSTACK_AUTH_TOKEN",
     ]
     env_vars = env_vars or os.environ
     env_vars = select_attributes(dict(env_vars), env_var_names)
@@ -477,42 +488,33 @@ def start_aws_auth_proxy_in_container(
         target_host = get_docker_host_from_container()
     env_vars["LOCALSTACK_HOST"] = target_host
 
-    # Use the Docker SDK command either if quiet mode is enabled, or if we're executing
-    # in Docker itself (e.g., within the LocalStack main container, as part of an init script)
-    use_docker_sdk_command = quiet or localstack_config.is_in_docker
-
     try:
         print("Proxy container is ready.")
-        command = f"{venv_activate}; localstack aws proxy -c {CONTAINER_CONFIG_FILE} -p {port} --host 0.0.0.0 > {CONTAINER_LOG_FILE} 2>&1"
-        if use_docker_sdk_command:
-            DOCKER_CLIENT.exec_in_container(
-                container_name,
-                command=["bash", "-c", command],
-                env_vars=env_vars,
-                interactive=True,
-            )
-        else:
-            env_vars_list = []
-            for key, value in env_vars.items():
-                env_vars_list += ["-e", f"{key}={value}"]
-            # note: using docker command directly, as our Docker client doesn't fully support log piping yet
-            command = [
-                "docker",
-                "exec",
-                "-it",
-                *env_vars_list,
-                container_name,
-                "bash",
-                "-c",
-                command,
-            ]
-            subprocess.run(command, stdout=sys.stdout, stderr=sys.stderr)
+        # Start proxy server in background using Python directly (no CLI dependency)
+        start_server_cmd = (
+            f"{venv_activate}; "
+            f'python -c "from aws_proxy.client.auth_proxy import AuthProxyAWS; '
+            f"AuthProxyAWS.start_from_config_file('{CONTAINER_CONFIG_FILE}', {port})\" "
+            f">> {CONTAINER_LOG_FILE} 2>&1 &"
+        )
+        # Wait for proxy server to start, then register with LocalStack
+        register_cmd = (
+            f"sleep 2 && {venv_activate}; "
+            f'python -c "from aws_proxy.client.auth_proxy import AuthProxyAWS; '
+            f"AuthProxyAWS.register_proxy('{CONTAINER_CONFIG_FILE}', {port}, '{target_host}')\" "
+            f">> {CONTAINER_LOG_FILE} 2>&1"
+        )
+        command = f"{start_server_cmd} {register_cmd}"
+        DOCKER_CLIENT.exec_in_container(
+            container_name,
+            command=["bash", "-c", command],
+            env_vars=env_vars,
+            interactive=False,
+        )
     except KeyboardInterrupt:
         pass
     except Exception as e:
         LOG.info("Error: %s", e)
-        if isinstance(e, subprocess.CalledProcessError):
-            LOG.info("Error in called process - output: %s\n%s", e.stdout, e.stderr)
     finally:
         try:
             if repl_config.CLEANUP_PROXY_CONTAINERS:

aws-proxy/aws_proxy/client/cli.py

@@ -1,11 +1,14 @@
 import re
 import sys
 
+# Note: This CLI plugin is deprecated and may be removed in a future version.
+# Use the LocalStack internal API or the LocalStack Web App to configure the proxy instead.
+
 import click
 import yaml
-from localstack.cli import LocalstackCli, LocalstackCliPlugin, console
-from localstack.pro.core.cli.aws import aws
-from localstack.pro.core.config import is_auth_token_configured
+from localstack_cli.cli import LocalstackCli, LocalstackCliPlugin, console
+from localstack_cli.pro.core.cli.aws import aws
+from localstack_cli.pro.core.config import is_auth_token_configured
 from localstack.utils.files import load_file
 
 from aws_proxy.shared.models import ProxyConfig, ProxyServiceConfig

aws-proxy/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 [project]
 name = "localstack-extension-aws-proxy"
 readme = "README.md"
-version = "0.2.3"
+version = "0.2.4"
 description = "LocalStack extension that proxies AWS resources into your LocalStack instance"
 authors = [
         { name = "LocalStack Team", email = "info@localstack.cloud"}