feat: add datumctl config, clusters and context

[gianarb]

This commit contains the foundation of being able to connect to multiple
Datum clusters and to switch in between projects and orgs via contexts.

This PR at the time you run this new version of the datumctl will create the
configuration file without breaking compatibility with your current authorized
login.

Every time you run login it will check if it knows the server you are connecting
to, if not it will add it to the list and create a context for you to use.

commands

A couple of new commands related to context and clusters are available under config subcommand.

Now that authenticated user are related to context I don't think we need the auth list and even less the auth switch in order to avoid confusions so I removed them

Limitation:

Currently, we do not have a way to discriminate when to reach the organization
control plane or the project one. It means that we can not accept project AND
org into the same context, only one can be set. Right now we prioritize project
because most like it is the one you want to use for the most.

User who wants to quickly switch in between org and project needs to create
multiple context.

Fixes datum-cloud/enhancements#653

[scotwells]

Thanks for putting this together. After reviewing, I want to propose a shift in how contexts get created.

Right now the path to having a working context is: login, then config set-cluster with a server URL, then config set-context wiring up the cluster + user + project, then config use-context.

That's a lot of manual setup before someone can be productive — especially since the orgs and projects already exist on the platform.

What if contexts were automatically discovered from the API instead of manually authored?

What I think the experience should feel like

First login — you're productive immediately:

$ datumctl login
Opening browser for authentication...
✓ Authenticated as Jane Doe (jane@acme.com)

You have access to 2 organizations:
  acme-corp (3 projects)
  personal  (1 project)

? Select a context to work in:
  acme-corp                         (org)
  acme-corp/web-app                 (project)
  acme-corp/billing-api             (project)
> acme-corp/infra                   (project)
  personal                          (org)
  personal/sandbox                  (project)

✓ Context set to acme-corp/infra

One command. Auth and context selection happen together. If the user only has one project, skip the picker and auto-select.

Day-to-day — no flags needed:

$ datumctl get dnszones
NAME          ZONE              AGE
marketing     marketing.acme    5d
api           api.acme.dev      12d

$ datumctl create -f manifest.yaml
dnszone.networking/staging created

$ datumctl mcp
Starting MCP server for acme-corp/infra...

Everything runs against the active context.

Checking where you are:

$ datumctl whoami
User:         Jane Doe (jane@acme.com)
Context:      acme-corp/infra
Organization: acme-corp
Project:      infra (proj-9e4d...)

Seeing who you're logged in as:

$ datumctl auth list
USER                        STATUS
jane@acme.com               Active
bob@acme.com

If you have logins across multiple endpoints, the endpoint column appears:

$ datumctl auth list
USER                        ENDPOINT                    STATUS
jane@acme.com               api.datum.net               Active
jane@acme.com               api.staging.datum.net
bob@acme.com                api.datum.net

Same for whoami — the endpoint line only appears when you have logins across multiple endpoints:

$ datumctl whoami
User:         Jane Doe (jane@acme.com)
Endpoint:     api.datum.net
Context:      acme-corp/infra
Organization: acme-corp
Project:      infra (proj-9e4d...)

Seeing what's available and switching:

$ datumctl ctx
ORGANIZATION   PROJECT        TYPE      CURRENT
acme-corp                     org
acme-corp      infra          project   *
acme-corp      web-app        project
acme-corp      billing-api    project
personal                      org
personal       sandbox        project

Switch to a project:

$ datumctl ctx use acme-corp/web-app
✓ Switched to acme-corp/web-app (project)

Switch to an org:

$ datumctl ctx use acme-corp
✓ Switched to acme-corp (org)

$ datumctl get projects
NAME           ID            AGE
infra          proj-9e4d     30d
web-app        proj-7f3a     25d
billing-api    proj-2b1c     10d

Or with an interactive picker when you don't pass an argument:

$ datumctl ctx use
? Select a context:
  acme-corp                         (org)
  acme-corp/web-app                 (project, current)
> acme-corp/infra                   (project)
  acme-corp/billing-api             (project)
  personal                          (org)
  personal/sandbox                  (project)

Overrides for scripts and CI:

$ datumctl get dnszones --project proj-other
$ DATUM_PROJECT=proj-other datumctl get dnszones

Active context stays the same — the override is per-invocation only. (The env vars would be new — not implemented yet — but would give CI and scripting a clean override without mutating config.)

Working at the org level:

Sometimes you want to work at the org scope rather than inside a specific project — listing projects, managing org-wide settings, etc.

$ datumctl ctx use acme-corp
✓ Switched to acme-corp (org)

$ datumctl get projects
NAME           ID            AGE
infra          proj-9e4d     30d
web-app        proj-7f3a     25d
billing-api    proj-2b1c     10d

Switching between accounts:

Each login session remembers its endpoint and last selected context. Switching users puts you right back where that user left off.

$ datumctl auth switch jane@acme.com
✓ Switched to Jane Doe (jane@acme.com)
  Context: acme-corp/infra

If the same email exists on multiple endpoints, the CLI prompts to disambiguate:

$ datumctl auth switch jane@acme.com
? Which login session?
> jane@acme.com (api.datum.net)
  jane@acme.com (api.staging.datum.net)

Multiple deployments (staging, self-hosted):

Most users just talk to production and never think about this. Each login is bound to a specific endpoint — datumctl login defaults to production, and datumctl login --endpoint <url> targets a different deployment. Contexts are always scoped to the active user's endpoint.

$ datumctl login --endpoint https://api.staging.datum.net
Opening browser for authentication...
✓ Authenticated as Jane Doe (jane@acme.com) on api.staging.datum.net
✓ Context set to acme-corp/infra

To get back to production, just switch users:

$ datumctl auth switch jane@acme.com
✓ Switched to Jane Doe (jane@acme.com)
  Endpoint: api.datum.net
  Context:  acme-corp/infra

How I think about the command surface

Command	What it does
datumctl login [--endpoint <url>]	Log in to an endpoint and select a context. Defaults to production.
datumctl logout [<email> | --all]	Remove credentials.
datumctl whoami	Show who you are and what context you're in.
datumctl auth list	List all authenticated users (shows endpoint column when multiple are in use).
datumctl auth switch <email>	Switch active user (and their endpoint + last context). Prompts if ambiguous.
datumctl ctx	List available contexts for the current user.
datumctl ctx use [<org/project>]	Switch context. Interactive picker if no argument.
datumctl auth get-token	Print access token (for kubectl/CI).
datumctl auth update-kubeconfig	Configure kubectl access.

All resource commands inherit the active context. --project and --organization remain as per-command overrides.

Context resolution order

1. --project / --organization flag (per-command)
2. DATUM_PROJECT / DATUM_ORGANIZATION env var (new — for CI/scripting)
3. Active context from local config
4. Clear error: No context set. Run 'datumctl ctx use' to select a project.

---

This user-experience should make it easier for users to immediately start interacting with their projects and organizations in datum cloud.

[gianarb]

Hello @scotwells ! wow there is a lot to unwrap here! I need to admit I already feel a bit confused about when I need to switch vs use. I see you put a of of emphasis on interactions like selections and so on. To be those are nice to have addition on top of a solid abstraction. I will get to it for sure but can you try to describe the hierarchy you are trying to build here? Because it looks a bit deeper compared to what I have in mind.

The Context struct is the core of the new config layer. It binds together three things:

• Cluster — which API server to talk to (references a named cluster entry)
• User — which authenticated identity to use (references a named user entry whose key maps to keyring-stored credentials)
• Namespace — the default namespace for requests (defaults to "default")
• ProjectID / OrganizationID — optional scope; mutually exclusive, a context can target either a single project or an entire organization

This idea of mutually exclusive is because currently we need to specify if we want to work at org spec or project spec as you said. But this keeps a pretty flat structure you have context, and just need to remember the name for them, you do not need to think about "I am in use mode so I am working at org level for this context or what?". Probably a bit less expressive but flat.

Contexts are stored by name in the config file (~/.datumctl/config) and one is marked as current-context. The full resolution chain is:

current-context → NamedContext → Context.Cluster → NamedCluster → Cluster.Server

  type Config struct {
      APIVersion     string
      Kind           string
      Clusters       []NamedCluster
      Users          []NamedUser
      Contexts       []NamedContext
      CurrentContext string
  }

  type NamedContext struct {
      Name    string
      Context Context
  }

  type Context struct {
      Cluster        string
      User           string
      Namespace      string
      ProjectID      string
      OrganizationID string
  }

  type NamedCluster struct {
      Name    string
      Cluster Cluster
  }

  type Cluster struct {
      Server                   string
      TLSServerName            string
      InsecureSkipTLSVerify    bool
      CertificateAuthorityData string
  }

  type NamedUser struct {
      Name string
      User User
  }

  type User struct {
      Key string
  }

onboarding

The current code translates what you have in your keychain as context, so the onboarding is transparent for current users. I am not sure if you want to keep that or just discard and force re-login. For new users or to creae new context the interactivity you suggested can be the default solution for sure. this helps humans no agents 😄

conclusion

Let me know what you think, as I said the UX on top or the interactive part can be a second step compared to how we want actually model the underline layer on disk. If you can help me understand a bit more what you have in mind there I will start from there.

[gianarb]

Your proposal with a double switch with context and use for org sounds a bit confusing but I will keep looking at it for a bit more