Reconsider AIContent as an abstract class

[stephentoub]

If a provider has an object representing content for which there's no abstraction, they can use their own AIContent-derived type to represent it. If they don't want to have a public AIContent-derived type, they could just have it be internal, at least exposing the RawRepresentation and AdditionalProperties for the consumer to query. But if they do that, it won't work with Native AOT, as there'd then be no way for the consumer to build knowledge of that AIContent into a JsonSerializerOptions with source generation.

We should consider making AIContent non-abstract and giving it a public instead of protected constructor. Then such a scenario would work, and unknown AIContent could deserialize as the base type.

[SteveSandersonMS]

I looked into this and am not sure that making AIContent have a public parameterless constructor will suffice to enable JSON serialization of arbitrary AIContent subclasses (note: it's already not abstract).

For example given:

private sealed class DerivedAIContent : AIContent;

I tried:

JsonSerializer.Serialize(new DerivedAIContent(), TestJsonSerializerContext.Default.Options);

... and it throws:

System.NotSupportedException : JsonTypeInfo metadata for type 'Microsoft.Extensions.AI.AIContentTests+DerivedAIContent' was not provided by TypeInfoResolver of type 'Microsoft.Extensions.AI.TestJsonSerializerContext'. If using source generation, ensure that all root types passed to the serializer have been annotated with 'JsonSerializableAttribute', along with any types that might be serialized polymorphically.

So it seems like unless you can reference DerivedAIContent in your JSO, it won't serialize regardless of whether AIContent is publicly constructable.

@stephentoub If you had some other usage pattern in mind, could you clarify?

[stephentoub]

So it seems like unless you can reference DerivedAIContent in your JSO

For Native AOT, any type you want to serialize/deserialize needs to be in the JSO, regardless of polymorphism, and it looks like you're testing this in our test project, which disables reflection-based serialization, hence why you're getting an NSE.

it won't serialize regardless of whether AIContent is publicly constructable.

It should. In addition to making it constructible, we'd also want to use [JsonPolymorphic] to fall back to the base type, e.g. this

using System.Text.Json;
using System.Text.Json.Serialization;

Console.WriteLine(JsonSerializer.Deserialize<MyBase>(JsonSerializer.Serialize<object>(new Derived1())));
Console.WriteLine(JsonSerializer.Deserialize<MyBase>(JsonSerializer.Serialize<object>(new Derived2())));
Console.WriteLine(JsonSerializer.Deserialize<MyBase>(JsonSerializer.Serialize<object>(new Derived3())));

class Derived1 : MyBase
{
    public string? Derived1Property { get; set; }
}

class Derived2 : MyBase
{
    public string? Derived2Property { get; set; }
}

class Derived3 : MyBase
{
    public string? Derived3Property { get; set; }
}

[JsonDerivedType(typeof(Derived1), typeDiscriminator: "first")]
[JsonDerivedType(typeof(Derived2), typeDiscriminator: "second")]
[JsonPolymorphic(IgnoreUnrecognizedTypeDiscriminators = true, UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToBaseType)]
class MyBase;

outputs:

Derived1
Derived2
MyBase

@eiriktsarpalis, thoughts?

[eiriktsarpalis]

[JsonPolymorphic(IgnoreUnrecognizedTypeDiscriminators = true, UnknownDerivedTypeHandling = JsonUnknownDerivedTypeHandling.FallBackToBaseType)]

Assuming the main reason users create their own derived content types is to attach more fields, I'm not sure how I feel about hardcoding a mode where such data is being silently discarded. If that happens it would typically indicate that the user forgot to register the derived type, no?

[stephentoub]

I'm more interested in someone being able to use the base type directly rather than having to create a custom derived type. If I'm a provider and I want to be able to surface all of my data, today my only option is to create custom AIContent-derived types, which then effectively requires that those types be publically exposed and puts a burden on the consumer to know about them and register them or else serialization will blow up. If a provider can instead just use the base type, putting the relevant underlying object into RawRepresentation (and possibly AdditionalProperties), they can still return such data to the consumer. The consumer can break glass and get at the data, without using needed to register anything and without needing the extra tyoes exposed public ally. This is really about minimizing the extra surface area alib needs to expose to a single method that returns an IChatClient.

[eiriktsarpalis]

Agreed. So it seems minimum action required is making that constructor public? Would we also need to expose additional properties to make the base type more useful/descriptive?

[stephentoub]

Agreed. So it seems minimum action required is making that constructor public? Would we also need to expose additional properties to make the base type more useful/descriptive?

Yes. That would enable the main thing I'm interested in enabling. We can discuss whether it also makes sense to add the [JsonPolymorphic] attribute separately if you're concerned about it.