build: Add per-step CPU and memory resource limits

[jirimoravcik]

Port of moby/buildkit#6569 to buildx. Adds --resource row with things like --memory, --memory-swap, --cpu-shares, --cpu-period, --cpu-quota, --cpuset-cpus, and --cpuset-mems to build, plus the equivalent bake target attributes

[crazy-max]

Thanks but I don't think Buildx should expose this as seven new top-level build flags and seven independent bake fields.

Can we make the Buildx API a single extensible resource collection instead? For CLI, I think --resource fits better with existing repeatable buildx flags:

--resource=memory=2g --resource=memory-swap=4g --resource=cpu-quota=50000

For bake, I think this should mirror that as one target field, like:

resources = ["memory=2g", "memory-swap=4g", "cpu-quota=50000"]

That keeps the user-facing API smaller, matches the existing repeatable key/value style, and avoids adding a new flag and bake field every time BuildKit grows another resource knob.

I also don't think we should add these to Compose x-bake. New Compose build fields should be added upstream in https://github.com/compose-spec/compose-spec first and implemented in https://github.com/compose-spec/compose-go, then Buildx can consume them from the normal Compose build model. Expanding x-bake for new fields moves the schema in the wrong place and we plan to deprecate it: #3025

[jirimoravcik]

Thanks but I don't think Buildx should expose this as seven new top-level build flags and seven independent bake fields.

Can we make the Buildx API a single extensible resource collection instead? For CLI, I think --resource fits better with existing repeatable buildx flags:

--resource=memory=2g --resource=memory-swap=4g --resource=cpu-quota=50000

For bake, I think this should mirror that as one target field, like:

resources = ["memory=2g", "memory-swap=4g", "cpu-quota=50000"]

That keeps the user-facing API smaller, matches the existing repeatable key/value style, and avoids adding a new flag and bake field every time BuildKit grows another resource knob.

I also don't think we should add these to Compose x-bake. New Compose build fields should be added upstream in https://github.com/compose-spec/compose-spec first and implemented in https://github.com/compose-spec/compose-go, then Buildx can consume them from the normal Compose build model. Expanding x-bake for new fields moves the schema in the wrong place and we plan to deprecate it: #3025

Thanks for the review. So the existing:

flags.StringVarP(&ignore, "memory", "m", "", "Memory limit")
	flags.MarkHidden("memory")

	flags.StringVar(&ignore, "memory-swap", "", `Swap limit equal to memory plus swap: "-1" to enable unlimited swap`)
	flags.MarkHidden("memory-swap")

	flags.Int64VarP(&ignoreInt, "cpu-shares", "c", 0, "CPU shares (relative weight)")
	flags.MarkHidden("cpu-shares")

	flags.Int64Var(&ignoreInt, "cpu-period", 0, "Limit the CPU CFS (Completely Fair Scheduler) period")
	flags.MarkHidden("cpu-period")

	flags.Int64Var(&ignoreInt, "cpu-quota", 0, "Limit the CPU CFS (Completely Fair Scheduler) quota")
	flags.MarkHidden("cpu-quota")

	flags.StringVar(&ignore, "cpuset-cpus", "", `CPUs in which to allow execution ("0-3", "0,1")`)
	flags.MarkHidden("cpuset-cpus")

	flags.StringVar(&ignore, "cpuset-mems", "", `MEMs in which to allow execution ("0-3", "0,1")`)
	flags.MarkHidden("cpuset-mems")

should stay there, right? or do we want to remove it as well?

[tonistiigi]

I think we want to keep the existing flags (and connect them with --resource) but leave them hidden. User's shouldn't use them, but if you have an old command there is no need to error.

For the bake format, I guess it should be a struct (or double format like the cache and output options).