What problem does it solve? Turning a working SageMaker demo endpoint into one you can leave running requires autoscaling, monitoring alarms, consistent tagging, and safe teardown — all of which are tedious to wire up correctly by hand and easy to get wrong (e.g., async scale-to-zero needs a separate step-scaling policy, and vLLM CUDA 13+ images fail silently without an InferenceAmiVersion). ## Core Features & Use Cases - Real-time deployment: deploy.py creates the model, endpoint config, and endpoint, then registers target-tracking autoscaling and three CloudWatch alarms (latency p99, 5XX errors, overhead latency) with consistent resource tags. - Async inference deployment: deploy_async.py supports scale-to-zero with a backlog target-tracking policy plus a step-scaling wake-from-zero policy, S3 output paths, and SNS notifications. - Smoke testing and teardown: invoke_endpoint.py invokes endpoints with BOM-safe UTF-8 payloads (avoiding the Windows PowerShell BOM gotcha), and teardown.py deletes alarms, autoscaling, endpoint, config, and model in safe order. - Use Case: Deploy a Qwen3 LLM on ml.g5.xlarge with a vLLM DLC image, autoscaling from 1–4 instances, then verify it actually serves requests by scanning CloudWatch logs for worker crashes before declaring success. ## Quick Start Ask the agent to deploy your model to a SageMaker real-time endpoint by providing the image URI, IAM role ARN, instance type, and region, then have it smoke-test the endpoint and print the teardown command.
