Handling shared models

[adamvandenhoven]

I don't know if a solution to this already exists, but I couldn't find anything obvious from my searching around, so I'm assuming its a new idea.

I have a number different apis that are all related but independent. In general, they have common schemas; for example several have Address schemas that are all the same.

If I happen to need two different APIs, both of which provide an Address, Kiota will create two different models, which just happen to be the same. If I want to write GeocodeAddress I'll either need two methods, one for each model, or I can use a partial to give them a common interface or super class (which means I'm manually repeating the same properties a third time). Neither is particularly joy inducing.

One thing that the Open API explicitly says is "An OpenAPI document MAY be made up of a single document or be divided into multiple, connected parts at the discretion of the user." The $ref, while typically just a fragment pointing to something in the current doc (#/components/schemas/Address), it can be a full url.

Given that x-ms-kiota-info can provide language level information (much of which is also settable as cli arguments), specifically the clientNameSpace, when kiota encounters "$ref": "https://api2.example.com/oas.yaml#/components/schema/Address" it could parse that and minimally add Address to the normal Models folder but with the namespace defined in the source schema doc. This way if I have two Apis that reference the same common schema, the partials will merge with no negligible impact (at least for C# and those that can handle partials).

Another solution (probably a more easily implemented) might be to retrieve and parse the linked schema for its namespace and if it exists, simply use the model name with the correct namespace and emit a message on completion like the prompt to run kiota info to run kiota generate for the referenced schemas (it may require the sorts of changes discussed in #4835 to work)

Because I may not have control over the referenced schemas, there is no guarantee that a x-ms-kiota-info will be defined in the referenced schema, an additional extension might be needed to provide this information something like:

components:
  x-ms-kiota-dependancies:
    "https://api2.example.com/oas.yaml": 
      languagesInformation:
        CSharp:
          clientNamespaceName: Example.Api2
          dependencyInstallCommand: dotnet add package {name} --version {version}
          pregenerated: true
          dependencies:
            - name: Example.Api2.Kiota
              version: 3.0.0

It occurs to me (yes, I'm creating solutions as I write) that this would allow you to generate the client without having to load and parse the referenced schema file since you can properly infer the model name from the schema path. Moreover, since I know that API2 has published its client as a nuget, I don't need kiota to generate one for me just to get its models, I can flag that with pregenerated and instead of saying "go generate this schema" you just include it with the info commands.

This last idea, using x-ms-kiota-dependancies might be the best solution to "how do I share models between apis" since its presence can trigger the behaviour, it wouldn't require any additional http requests (a potentially big problem is api1 references api2 which references api3....) and it should let circular references just work?

[baywet]

Hi @adamvandenhoven
Thank you for using kiota and for reaching out with such a detailed topic.

Something that wasn't clear to me from reading the question: are all those APIs on the same host / fqdn? If so why not simply generating/authoring an aggregated document?

Assuming the models are truely identical on the backend (not a facet design pattern) and they are on different hosts, there's no way to reduce duplication today.
This has to do with the fact that the fluent API depends on the models, namespaces might be different etc...
It's also highly dependent on the language.

In my opinion that's mostly a good thing. Generated clients are infrastructure services (DDD) and you generally will want to interface them with more central layers. So having potentially duplicated models that belong to different services is not necessarily a bad thing in that context.

That being said, it might be important for some scenarios. You might be able to get around this limitation by doing something similar to what we do in TypeScript:

1. Generate all your clients.
2. Move all the models to a common package.
3. Update the imports in the fluent API.

This will require a bit of scripting but should eventually work.

Let us know if you have additional comments or questions.

[adamvandenhoven]

1. Some of our APIs are on the same fqdn since they're currently behind apim, but not all, and there is not guarantee that its progressing one way or the other.
2. We have situations where two apis return the same schema because the firsts calls the second, and its likely that some clients will call both, "why is this from that api not the same as it is from the other since its the same data?"
3. Our APIs are maintained (and used) by different teams with different levels of skill and maturity, providing common building blocks and a way to be consistent with them is useful. I would love to be able to make the instructions as simple as "take the schema doc you want want kiota and follow the on screen instructions"
4. I'm not sure I agree that having the same shape of object realized as different classes is actually desirable, unless the apps using the clients are never marshalling data from one to the other. And I'm not sure I could articulate to another developer why the address they got from the JobApplication API can't simply be passed to the Employee API (to pick a trivial example).
5. Personally (and I realize that this a minority opinion) I dislike the fact that all we ever do is send and receive arbitrary JSON and rely on knowing that "this URL returns things with that shape". I would much prefer to have application/vnd.example.address+json and know that the shape for that is always the same thing. Yes, I've read and understood too much of Roy Fielding's dissertation. What I am suggesting is definitely informed by that, but I still think its valuable.

I can conceive of many different ways to make do with what Kiota already does; for example, I could use C# operator overloading to create implicit casts between the different types.

If kiota already handles external schema references, then I have to assume that you're also parsing and honouring the x-ms-kiota-info that could be in that doc (which is my first suggestion), if Kiota does NOT handle external schema refs, then you're not supporting a non-trivial portion of the open api spec. I'm suggesting one way to make that work.

[baywet]

Thank you for the additional information.

I dislike the fact that all we ever do is send and receive arbitrary JSON and rely on knowing that "this URL returns things with that shape"

Agreed, but unfortunately this is where the industry is today, and I don't see much traction in addressing that. But even if the media type were more explicit, the generated code still needs to accept and return a specific type for statically typed languages. So I'm expecting coupling to remain.

external refs
Not supported today, we're waiting on our dependency to add support for it. Even assuming kiota supports it, it'd project one model per client even if the external reference is the same for multiple clients. So I don't think it'd address your current concern.

Besides scripting to publish your own deduplicated clients or using implicit conversions (not supported by all languages), I think the duplication and mapping will be a tradeoff for the simplicity of "just generating a client".

Let us know if you have any additional comments or questions.

[adamvandenhoven]

Well I tried. I think it would have been a useful feature, particularly since it could have allowed for a choice in how models are created, but I get the reasoning. I can create work arounds for what I need.