Blog Post: Introducing JobSet #45759

danielvegamyhre · 2024-04-02T18:42:47Z

We would like to publish a blog post introducing JobSet, a K8s native API for distributed ML training and HPC workloads.

cc @ahg-g @kannon92 I think we still need to align on one example and ideally make it more concrete and polished. We should also explain the user story above it.

kannon92 · 2024-04-02T18:44:53Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+
+## Why JobSet?
+
+The Kubernetes community’s recent enhancements to the batch ecosystem on Kubernetes attracted ML engineers who have found it to be a natural fit for the requirements of running distributed training workloads. Large ML models (particularly LLMs) which cannot fit into the memory of the GPU or TPU chips on a single host are often distributed across tens of thousands of accelerator chips, which in turn may span thousands of hosts. 


Please add some new lines. (I can only comment on this entire paragraph).

With this, I try to follow a newline per sentence.

Hmm this paragraph is only 2 sentences. Do we really need/want each sentence to be it's own paragraph? In the Kueue blog post they use paragraphs: https://kubernetes.io/blog/2022/10/04/introducing-kueue/

I think it will be rendered the same. But if you have no new lines, people can only suggest the entire paragraph.

Localization teams ask for Markdown where the source is wrapped at around 100 characters.

Please do that.

Updated so column width is max of ~100 chars (except in long links)

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

netlify · 2024-04-02T18:45:44Z

✅ Pull request preview available for checking

Built without sensitive environment variables

Name	Link
🔨 Latest commit	`d3d0821`
🔍 Latest deploy log	https://app.netlify.com/sites/kubernetes-io-main-staging/deploys/66590787b28a48000881c1a9
😎 Deploy Preview	https://deploy-preview-45759--kubernetes-io-main-staging.netlify.app
📱 Preview on mobile	Toggle QR Code... Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify site configuration.

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

kannon92 · 2024-04-02T18:50:53Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+
+**Authors**: Daniel Vega-Myhre (Google), Abdullah Gharaibeh (Google), Kevin Hannon (Red Hat)
+
+In this article, we introduce [JobSet](https://jobset.sigs.k8s.io/), an open source API for representing distributed jobs. The goal of JobSet is to provide a stable API for running/building APIs with AI/ML and HPC use cases in mind.


I struggle if we should define HPC..

What do you mean by "running/building APIs"?

I think HPC is a known term, but I also don't understand the running/building APIs part.

…2-introducing-jobset.md Co-authored-by: Kevin Hannon <[email protected]>

haircommander · 2024-04-02T19:53:39Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+ command:
+ - bash
+ - -xc
+ - |
+ sleep 10000
+ - name: workers


are there any tangible examples of this? I assume not if you're using bash here, but I find it odd this is abstract

I agree, we should make the example more concrete.

The examples I know of require TPUs/GCP and I am not sure if we can do that for a general k8s blog post. So we decided to do simple options that don't require accelerators or lock you into a specific vendor.

I think we can probably come up with a GPU based example and simply use fake node labels like "cloud.provider.com/gpu-label" instead of a vendor specific one

Maybe you can link to GKE related use cases as concrete examples?

If @haircommander point is taking literal, we should show concrete usecases also.

What do you think about that @sftim? Should we try to keep the post self contained?

kannon92 · 2024-04-02T19:53:57Z

/cc @haircommander

Trying to find an impartial reviewer ;)

haircommander · 2024-04-02T19:54:42Z

I have one note but I found this informative while doing a good job of laying the groundwork needed.

LGTM (assuming the note is unaddressable)

sftim

Some partial feedback (not yet a full review)

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

sftim · 2024-04-02T20:06:09Z

content/en/blog/_posts/2024-04-02-introducing-jobset/jobset_diagram.png

We prefer SVG:

easier to localize

scalable

Could you draw this as SVG instead?

Replaced png diagram with svg version

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

sftim · 2024-04-02T20:08:19Z

(if this is not yet ready for review by the blog team, please change the title to start with [WIP])

sftim · 2024-04-02T20:08:39Z

/hold

pending assignment of publication date

danielvegamyhre · 2024-04-02T20:13:21Z

(if this is not yet ready for review by the blog team, please change the title to start with [WIP])

Thanks Tim, added [WIP] to the title.

ahg-g · 2024-04-03T05:42:50Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+
+**Authors**: Daniel Vega-Myhre (Google), Abdullah Gharaibeh (Google), Kevin Hannon (Red Hat)
+
+In this article, we introduce [JobSet](https://jobset.sigs.k8s.io/), an open source API for representing distributed jobs. The goal of JobSet is to provide a stable API for running/building APIs with AI/ML and HPC use cases in mind.


I think HPC is a known term, but I also don't understand the running/building APIs part.

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

sftim · 2024-04-03T08:43:34Z

I propose the 26th of April as publication date. Does that work?

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

sftim · 2024-05-03T11:02:35Z

Let's pick a new publication date. How about the 7th of May?

sftim · 2024-05-03T11:03:03Z

You should remove [WIP] from the PR title @danielvegamyhre if / when you think this is ready to be reviewed.

ahg-g · 2024-05-31T17:40:05Z

/lgtm

Thanks @danielvegamyhre !

sftim

Publication date is now in the past

/lgtm cancel

sftim · 2024-06-23T15:32:30Z

LGTMs from #45759 (comment) and #45759 (comment) mean that all this needs is a new publication date.

Sorry about the wait; we have very few active blog reviewers.

I suggest aiming for the 17th of July; does that work?

sftim · 2024-06-23T15:33:26Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+accelerators, without resorting to scripting or Helm charts to generate many versions of the
+same job but with different names.
+
+![JobSet Architecture Diagram](jobset_diagram.svg "JobSet Architecture")


nit: using a figure shortcode, with a caption, is even better. You can also set the diagram-medium class; I recommend doing that.

sftim

This article introduces an unregistered official annotation.

You / WG Batch should register it, especially if any in-project code already uses the annotation.
(you do that by updating https://kubernetes.io/docs/reference/labels-annotations-taints/)
We can either register the annotation before publication, which is pretty straightforward, or we - blog editors - can add an editors' note that the annotation is not yet official.

I'm opposed to publicising unregistered annotations because if we do it once, people will keep asking for a similar exception.

sftim · 2024-06-23T15:35:25Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+TPU multislice training out-of-the-box with very little configuration required by the user.
+
+```yaml
+# Run a simple Jax workload on 


Did you mean

Suggested change

# Run a simple Jax workload on

# Run a simple Jax workload on Google Cloud

?

sftim · 2024-06-23T15:35:28Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

+ name: multislice
+ annotations:
+ # Give each child Job exclusive usage of a TPU slice 
+ alpha.jobset.sigs.k8s.io/exclusive-topology: cloud.google.com/gke-nodepool


We should register (and, ideally, deprecate) this annotation before we publicize it.
The page to register it on is https://kubernetes.io/docs/reference/labels-annotations-taints/

It needs registering because although it uses a deprecated pattern (“alpha” in the key), it's inside an official Kubernetes namespace for annotations.

If WG Batch do deprecate this annotation and switch to something different, that's a nice but separate improvement.

We still need the original one registered if any released code inside https:/kubernetes/ uses it, or previously used it.

I don't know if we should register alpha annotations/labels. Sounds like drift waiting to happen.

I don't know if most sig projects register the labels/annotations on that website.

Marking annotations as alpha or beta is deprecated; we should eventually just use the plain form (eg jobset.kubernetes.io/exclusive-topology). AIUI if a SIG wants to mark something as experimental, and for that not to be registered, it needs to be outside kubernetes.io or k8s.io.

We should register this one, or pick something different and blog about the updated annotation.

@danielvegamyhre WDYT? Could we have a post without this annotation for now? I know that changing these annotations could break users.

@ahg-g if you're wondering how much work is involved in registering the annotation, it's a 10 line PR to update https://kubernetes.io/docs/reference/labels-annotations-taints/

Please consider that if writing >= 10 lines of objection.

I am fine with registering it if that addresses your concern, I was just asking if we are allowed to register annotations that are not part of upstream k8s.

Got it! The way I've done it, anything with an in-project origin can be registered (even if it shouldn't have been used; as soon as code ships that might use the annotation, we have to document it).

Registered means “make a record that it's already in use“; it's not like an API review - deliberately!

https://kubernetes.io/docs/reference/labels-annotations-taints/

I can do this

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

sftim · 2024-06-23T15:36:55Z

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md

Please wrap the markdown at 100 or so (source) characters per line.

Once you've done that, I recommend squashing commits.

k8s-ci-robot · 2024-06-23T15:39:34Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from sftim. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

content/en/blog/OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

sftim · 2024-06-24T09:25:00Z

Content LGTMs noted, but publication date does need to change.

/lgtm cancel

sftim · 2024-08-06T16:54:32Z

There's a long disagreement over whether it's OK to publish this, given it would tacitly bless using annotations the wrong way (the JobSet controller looks for annotations and labels that look Kubernetes official, but aren't yet on our list of known annotations).

I'm personally loth to publish this without settling that disagreement. Making an exception once has me thinking it could set a precedent we end up having to live with.

danielvegamyhre · 2024-08-24T23:54:37Z

There's a long disagreement over whether it's OK to publish this, given it would tacitly bless using annotations the wrong way (the JobSet controller looks for annotations and labels that look Kubernetes official, but aren't yet on our list of known annotations).

I'm personally loth to publish this without settling that disagreement. Making an exception once has me thinking it could set a precedent we end up having to live with.

I'm fine with registering the annotations before publishing

danielvegamyhre · 2024-09-08T18:11:56Z

Update: we are close to merging the PR registering JobSet labels/annotations (#47383). Once that is done we can move forward with publishing the blog post here.

jobset blog post

a5abcb9

k8s-ci-robot requested review from Gauravpadam and nate-double-u April 2, 2024 18:42

kannon92 reviewed Apr 2, 2024

View reviewed changes

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md Outdated Show resolved Hide resolved

kannon92 reviewed Apr 2, 2024

View reviewed changes

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md Show resolved Hide resolved

kannon92 reviewed Apr 2, 2024

View reviewed changes

content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-02-introducing-jobset.md Outdated Show resolved Hide resolved

kannon92 reviewed Apr 2, 2024

View reviewed changes

Update content/en/blog/_posts/2024-04-02-introducing-jobset/2024-04-0…

dafd037

…2-introducing-jobset.md Co-authored-by: Kevin Hannon <[email protected]>

haircommander reviewed Apr 2, 2024

View reviewed changes

k8s-ci-robot requested a review from haircommander April 2, 2024 19:53

danielvegamyhre added 2 commits April 2, 2024 20:02

add future work section

536922a

remove nccl and xla references

95c884e

sftim reviewed Apr 2, 2024

View reviewed changes

k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Apr 2, 2024

100 char max line length

66759c0

danielvegamyhre changed the title ~~Blog Post: Introducing JobSet~~ [WIP] Blog Post: Introducing JobSet Apr 2, 2024

k8s-ci-robot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Apr 2, 2024

address comments

4ef3d34

danielvegamyhre added 2 commits April 2, 2024 20:32

replace png with svg

a6383c0

use description lists

08224a2

ahg-g reviewed Apr 3, 2024

View reviewed changes

Ritikaa96 reviewed Apr 4, 2024

View reviewed changes

add tpu multislice example and address comments

d3d0821

k8s-ci-robot assigned ahg-g May 31, 2024

k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label May 31, 2024

danielvegamyhre changed the title ~~[WIP] Blog Post: Introducing JobSet~~ Blog Post: Introducing JobSet May 31, 2024

k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label May 31, 2024

sftim reviewed Jun 1, 2024

View reviewed changes

k8s-ci-robot assigned sftim Jun 1, 2024

k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 1, 2024

k8s-ci-robot requested a review from ahg-g June 1, 2024 20:20

sftim reviewed Jun 23, 2024

View reviewed changes

sftim requested changes Jun 23, 2024

View reviewed changes

danielvegamyhre mentioned this pull request Aug 25, 2024

Register JobSet labels, annotations, and taints #47666

Closed

danielvegamyhre mentioned this pull request Oct 4, 2024

Document labels, annotations and taints for JobSet #47383

Open


		## Why JobSet?

		The Kubernetes community’s recent enhancements to the batch ecosystem on Kubernetes attracted ML engineers who have found it to be a natural fit for the requirements of running distributed training workloads. Large ML models (particularly LLMs) which cannot fit into the memory of the GPU or TPU chips on a single host are often distributed across tens of thousands of accelerator chips, which in turn may span thousands of hosts.


		Authors: Daniel Vega-Myhre (Google), Abdullah Gharaibeh (Google), Kevin Hannon (Red Hat)

		In this article, we introduce [JobSet](https://jobset.sigs.k8s.io/), an open source API for representing distributed jobs. The goal of JobSet is to provide a stable API for running/building APIs with AI/ML and HPC use cases in mind.

	# Run a simple Jax workload on
	# Run a simple Jax workload on Google Cloud

Blog Post: Introducing JobSet #45759

Are you sure you want to change the base?

Blog Post: Introducing JobSet #45759

Conversation

danielvegamyhre commented Apr 2, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

netlify bot commented Apr 2, 2024 • edited Loading

✅ Pull request preview available for checking

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kannon92 commented Apr 2, 2024

haircommander commented Apr 2, 2024

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim commented Apr 2, 2024

sftim commented Apr 2, 2024

danielvegamyhre commented Apr 2, 2024

Choose a reason for hiding this comment

sftim commented Apr 3, 2024

sftim commented May 3, 2024

sftim commented May 3, 2024

ahg-g commented May 31, 2024

sftim left a comment

Choose a reason for hiding this comment

sftim commented Jun 23, 2024

Choose a reason for hiding this comment

sftim left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

sftim Aug 5, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

k8s-ci-robot commented Jun 23, 2024

sftim commented Jun 24, 2024

sftim commented Aug 6, 2024

danielvegamyhre commented Aug 24, 2024

danielvegamyhre commented Sep 8, 2024

netlify bot commented Apr 2, 2024 •

edited

Loading

sftim Aug 5, 2024 •

edited

Loading