Update sync for credentials

[josephjclark]

This PR updates credential tracking in a new project to use a credential id, rather than the UUID.

This has several benefits:

• I never want to show a user a UUID, as a general point of principle
• When reading a project.yaml, it's easier to understand which credential a step is using
• credentials.yaml is easier to write and maintain when using UUIDs, rather than UUIDs
• When deploying a project to a new instance, credentials will track correctly so long as a credential with the same name and owner exists on that instance.

how this works is the project.yaml will track all attached credentials for that project on that instance, mapping the id to a UUID. Just like a v1 state file does. The workflow.yaml only contains the id. When syncing to that instance, the CLI will map the credential id to a UUID and send it to the provisioner.

There are some problems I see which I think we'll come back to later:

• user emails are in the project.yaml files, and I understand this to be problematic
• when deploying a project to an instance, the credential will only be tracked if the owner is the same. The same email address must be used to set up all instances. This is probably fine but I expect will bite us one day.

Both problems can be overcome if we allow a credential to have an explicit unique id that can be set in the app. Default it to owner-name by all means - but allow it to be changed, so that it can be standardised across instances (and have sensitive information removed).

Fixes #1254

TODO:

• Add support for ids in credentials.yaml
• manually test a new project against production
• When merging projects, if not a sandbox merge, generate new UUIDs for new credentials on the target
• manually test adding a new credential against production

Known Issues

• If your local credential id fails to resolve, you'll get a vague 500 error back from lightning
• Works great on new projects - but for existing projects you have to add a credential to the project AND fetch the latest yaml BEFORE you can add it to the local step. Else you'll get a 500 error

QA

• When fetching, project yamls should include a credentials array, and also use credential ids internally
• workflow yaml should contain credential ids now (not uuids)
• Credential maps should work with the new Ids (the old UUID format is broken)
• When creating a new project with deploy, if mapped to an existing credential, the credential should be linked in the app
• When adding a new credential to an existing project, on deploy the credential should get added

AI Usage

Please disclose whether you've used AI anywhere in this PR (it's cool, we just
want to know!):

• I have used Claude Code
• I have used another model
• I have not used AI

You can read more details in our
Responsible AI Policy

[josephjclark]

@midigofrank this PR will mean that CLI users can attach any credential to a project (so long as they know the name and owner)

Can I just check that the Provisioner has proper security and won't attach a credential to a project unless the PAT being used has the appropriate permissions? Like I can't attach a credential that you own with my PAT

[josephjclark]

This works pretty well in manual tests

✅ Take an existing project and run openfn project deploy --new, and the credential will be mapped if owner and name are the same
✅ Add a credential by id to step in an existing project and openfn project deploy. The credential will be attached

Problems though:
❌ Super unhelpful 500 error the credential fails to map and sends an invalid value to lightning
❌ When adding a new credential to a step, you have to add the credntial to the project in the app, then openfn project fetch to get the UUID. And then it'll work.

Both can be fixed but will take time I don't have.

EDIT: I had a brainwave and both issue should now be FIXED

[midigofrank]

@josephjclark yes, you cannot use someone else's credential. Your PAT only gets you access to your credentials or credentials that already exist in the project