AWS Batch
An AWS service that runs big piles of jobs and rents the machines only while they work.
AWS Batch runs batch jobs: tasks that do not need to answer a user right away, but need a lot of computing. Rendering the frames of an animation, running thousands of genome analyses, recalculating every risk model overnight, processing a month of sensor data. You send the jobs, Batch figures out where and when to run them.
You package each job as a container and submit it to a job queue. Batch looks at the queue and at the resources each job asks for, such as CPUs, memory or GPUs, and starts the right amount of capacity in a compute environment: EC2 instances, including cheaper Spot instances, or Fargate. When the queue empties, it scales back down so you are not paying for idle machines. Jobs can have priorities and can depend on each other, so step B waits until step A finishes.
Before services like this, a lab or company had to buy and maintain its own cluster plus scheduling software, and the cluster sat idle between big runs. Batch itself has no extra charge; you pay for the compute it launches.
Keep in mind
- Built for jobs that need lots of compute but not an instant answer.
- Jobs are containers submitted to queues, with priorities and dependencies.
- Scales EC2, Spot or Fargate capacity up and down with the queue.
- No charge for Batch itself; you pay for the compute it uses.
