How to Set an existing ECS cluster to a compute environment in AWS Batch

问题 我在我的环境中运行了一个ECS cluster ,它配置了任务、存储库和服务以及自动缩放。 我正在发现AWS Batch服务,当我创建一个新的compute environment时,一个新的集群被添加到ECS服务中。 我想知道如何为我现有的集群设置compute environment ? 回答1 我知道这是一个很老的问题,而且情况并不完全相同,但我正在努力解决类似的问题,所以它可能对某人有帮助。 在我的情况下,Batch 覆盖了我的ECS_DISABLE_IMAGE_CLEANUP设置,导致我特殊的专用超高优先级作业队列(及其计算环境)丢失其图像,当它们必须再次被拉出时会导致令人讨厌的延迟。 为了解决这个问题,我必须创建一个启动模板(在 EC2 控制台中),将所有内容设置为“不要更改”,以及一些额外的“用户数据”(在高级部分的末尾): MIME-Version: 1.0 Content-Type: multipart/mixed; boundary="==MYBOUNDARY==" --==MYBOUNDARY== Content-Type: text/x-shellscript; charset="us-ascii" #!/bin/bash echo ECS_DISABLE_IMAGE_CLEANUP=true>>/etc/ecs/ecs.config --=

2022-07-11 10:01:38    分类:技术分享    amazon-web-services   cluster-computing   solution   amazon-ecs

Running multiple ECS tasks based on same task definitions in one host, using different ports

问题 我有一个 ecs 任务定义。 上述任务定义的多个任务可以在多个端口上的一个 EC2 实例上运行吗? 我已经在多个 ec2 实例上运行了多个任务。 我想减少资源的消耗。 因为一个 EC2 实例有足够的资源来运行多个任务。 是否可以在一个 EC2 实例的不同端口上多次运行相同的任务? 回答1 是的,ECS 自 2016 年以来对此提供了很好的支持。您可以在容器定义中将主机端口留空,这将导致为您的容器选择一个随机端口。 因此,同一任务定义的更多实例可以在一个 ECS 实例上运行。 您可以将 ECS 服务与 Application Load Balancer 结合使用,以便在启动新任务时将端口号注册到关联的目标组中。 这样您就不必处理随机端口。 如果您通过 AWS 控制台设置服务,则配置非常简单。 回答2 这可以通过在定义任务时将容器定义的端口映射设置中的主机端口设置为 0 来配置。 以下设置在任务的容器定义中可用。 它允许 ECS 将随机端口分配给在同一 EC2 中运行的任务。 更多详情请查看 - 为 ECS 设置动态端口映射

2022-05-15 17:47:20    分类:技术分享    amazon-web-services   amazon-ec2   load-balancing   amazon-ecs

AWS application load balancer health check is failing with ECS instance due to incorrect Security Group

I'm running an ECS cluster with ECS service to run a container in it. I'm also using an AWS application load balancer that points to this ECS instance. The container getting stopped every time because of health check with failure with exit code 143. Task failed ELB health checks in (target-group arn:aws:elasticloadbalancing:us-east-1:426955121075:targetgroup/cs1-TargetG-UG5G7MZZLBOA/246895af7886b697) I debugged a lot found that that Application Load Balancer (ALB) could not reach to my ECS instance due to issue with the assigned Security Group rules attached with the ECS instance. In my

2022-05-03 07:09:35    分类:问答    amazon-web-services   amazon-ecs   aws-application-load-balancer

ECS - communication between tasks

I am trying to deploy 2 containers on 2 different tasks (1 container per task), one is my frontend and the other is my backend server. I am trying to figure out how to configure the communication between them. I saw that a load balancer in a service is a good option. However, should I configure load balancer for my front end server and another one for my backend? Meaning each time I have public-facing services and private services I need 2 load balancers? I would like to only expose my front-end to the public internet and my backend will remain private (although I make API requests to the

2022-05-03 06:43:35    分类:问答    load-balancing   amazon-ecs

How to decide which ECS tool to use?

I am at a bit of a crossroads here. My goal is to automate creating my ECS architecture and deploying my docker-compose services to ECS Fargate, but there are so many ways to do it! Hoping to get some insight from the community on picking the right tool for the job. What are the use cases for each of these? When should I pick one over the other? Docker ECS integration https://docs.docker.com/cloud/ecs-integration/ ecs-cli https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ECS_CLI.html AWS Copilot https://docs.aws.amazon.com/AmazonECS/latest/developerguide/AWS_Copilot.html AWS CLI ecs

2022-05-03 01:58:30    分类:问答    amazon-web-services   docker   docker-compose   amazon-ecs

AWS CLI 2 can't update-service use CLI

I have a cluster on ecs everything works well! When I used aws cli v.1, I could update my service used a command like this aws ecs update-service --cluster [cluster-name] --service [service-name] --task-definition [task-name] --force-new-deployment. After updating CLI to v.2 I try to use this command and everything just stuck! I didn't find any changes in aws documentation. Do you have any ideas? update: my screenshot the problem is that everything starts well, without errors or warnings, it just gets stuck!

2022-05-02 14:33:41    分类:问答    amazon-ecs

Error with Jenkins ECS and EFS

问题 我正在尝试使用 Jenkins 部署到带有 EFS 的 ECS。 (/var/jenkins_home 已安装) 似乎一切都很好,除了当我部署服务时,我在日志中收到以下错误: touch: cannot touch ‘/var/jenkins_home/copy_reference_file.log’: Permission denied Can not write to /var/jenkins_home/copy_reference_file.log. Wrong volume permissions? No newer events found at the moment. Retry. 有没有办法解决这个问题,所以我对/var/jenkins_home有足够的权限? 回答1 是的,它有点疯狂,但一个简单的解决方案就是在你的 efs 文件位置运行 chown 命令。 例如,如果 '/efs' 是您的 EFS 挂载,则简单运行sudo chown 1000 efs/ 应该可以立即正常工作 回答2 只是添加@jonathan答案创建一个新的docker文件,它会像这样覆盖jenkins官方图像 from jenkins/jenkins USER root RUN chown -R 1000 /var 创建新的 docker 镜像docker build

2022-05-02 03:50:21    分类:技术分享    amazon-web-services   jenkins   amazon-ecs   amazon-efs

During Terraform destroy, terraform is trying to destroy the ECS cluster before destroying the Auto-scaling group and is failing

I have used ECS with capacity provider for deployment of my application and have enabled scale-in protection for ASG used by capacity provider. During Terraform destroy I see terraform trying to destroy ECS cluster and after trying for 10 minutes it fails and outputs, Error: Error deleting ECS cluster: ClusterContainsContainerInstancesException: The Cluster cannot be deleted while Container Instances are active or draining. What i'm I doing wrong here, Relevant Terraform script, FOR ECS #ecs auto-scaling resource "aws_appautoscaling_target" "ecs_target" { max_capacity = var.ecs_max_size -- (8)

2022-05-02 03:24:13    分类:问答    amazon-web-services   terraform   amazon-ecs   terraform-provider-aws

Nginx static files 404 with django swagger rest api

I have Django rest API application and swagger for that is working fine locally. I am trying to configure it using containers and deploy it on ECS. Now when I build and run the container the application works fine (I mean the swagger UI is appearing). When I attach the application load balancer to it, on the browser it is giving me 404 files not found for js and CSS files. Here is the console output from the web browser. Here is my Nginx config file # nginx.default add_header X-Content-Type-Options nosniff; include /etc/nginx/mime.types; add_header X-XSS-Protection "1; mode=block"; server {

2022-05-01 22:45:51    分类:问答    django   amazon-web-services   nginx   django-rest-framework   amazon-ecs

Is it possible to trigger an AWS Fargate task upon an item being added to an SQS queue?

For clarification, what I'm trying to do is fire off a Fargate task when theres an item in a specific queue. I've used this tutorial to get pretty much where I am. This worked fine but the problem I ran into was every file upload (the structure of the s3 bucket is s3_bucket_name/{unknown_name}/known_file_names) was resulting in a task being triggered and I only want/need it to trigger once per {unknown_name} . I've since changed my configuration to add an item to a queue when it detects a test_file.txt file. Is it possible to trigger a fargate task on a queue like this? If so, how?

2022-05-01 20:13:42    分类:问答    amazon-web-services   amazon-s3   terraform   amazon-ecs   amazon-sqs