Querying Semantic Memory with XAF and the DevExpress Chat Component

by Joche Ojeda | Oct 21, 2024 | A.I, Semantic Kernel

A few weeks ago, I received the exciting news that DevExpress had released a new chat component (you can read more about it here). This was a big deal for me because I had been experimenting with the Semantic Kernel for almost a year. Most of my experiments fell into three categories:

1. NUnit projects with no UI (useful when you need to prove a concept).
2. XAF ASP.NET projects using a large textbox (String with unlimited size in XAF) to emulate a chat control.
3. XAF applications using a custom chat component that I developed—which, honestly, didn’t look great because I’m more of a backend developer than a UI specialist. Still, the component did the job.

Once I got my hands on the new Chat component, the first thing I did was write a property editor to easily integrate it into XAF. You can read more about property editors in XAF here.

With the Chat component property editor in place, I had the necessary tool to accelerate my experiments with the Semantic Kernel (learn more about the Semantic Kernel here).

The Current Experiment

A few weeks ago, I wrote an implementation of the Semantic Kernel Memory Store using DevExpress’s XPO as the data storage solution. You can read about that implementation here. The next step was to integrate this Semantic Memory Store into XAF, and that’s now done. Details about that process can be found here.

What We Have So Far

1. A Chat component property editor for XAF.
2. A Semantic Kernel Memory Store for XPO that’s compatible with XAF.

With these two pieces, we can create an interesting prototype. The goals for this experiment are:

1. Saving “memories” into a domain object (via XPO).
2. Querying these memories through the Chat component property editor, using Semantic Kernel chat completions (compatible with all OpenAI APIs).

Step 1: Memory Collection Object

The first thing we need is an object that represents a collection of memories. Here’s the implementation:

[DefaultClassOptions]
public class MemoryChat : BaseObject
{
    public MemoryChat(Session session) : base(session) {}

    public override void AfterConstruction()
    {
        base.AfterConstruction();
        this.MinimumRelevanceScore = 0.20;
    }

    double minimumRelevanceScore;
    string name;

    [Size(SizeAttribute.DefaultStringMappingFieldSize)]
    public string Name
    {
        get => name;
        set => SetPropertyValue(nameof(Name), ref name, value);
    }

    public double MinimumRelevanceScore
    {
        get => minimumRelevanceScore;
        set => SetPropertyValue(nameof(MinimumRelevanceScore), ref minimumRelevanceScore, value);
    }

    [Association("MemoryChat-MemoryEntries")]
    public XPCollection<MemoryEntry> MemoryEntries
    {
        get => GetCollection<MemoryEntry>(nameof(MemoryEntries));
    }
}

This is a simple object. The two main properties are the MinimumRelevanceScore, which is used for similarity searches with embeddings, and the collection of MemoryEntries, where different memories are stored.

Step 2: Adding Memories

The next task is to easily append memories to that collection. I decided to use a non-persistent object displayed in a popup view with a large text area. When the user confirms the action in the dialog, the text gets vectorized and stored as a memory in the collection. You can see the implementation of the view controller here.

Let me highlight the important parts.

When we create the view for the popup window:

private void AppendMemory_CustomizePopupWindowParams(object sender, CustomizePopupWindowParamsEventArgs e)
{
    var os = this.Application.CreateObjectSpace(typeof(TextMemory));
    var textMemory = os.CreateObject<TextMemory>();
    e.View = this.Application.CreateDetailView(os, textMemory);
}

The goal is to show a large textbox where the user can type any text. When they confirm, the text is vectorized and stored as a memory.

Next, storing the memory:

private async void AppendMemory_Execute(object sender, PopupWindowShowActionExecuteEventArgs e)
{
    var textMemory = e.PopupWindowViewSelectedObjects[0] as TextMemory;
    var currentMemoryChat = e.SelectedObjects[0] as MemoryChat;

    var store = XpoMemoryStore.ConnectAsync(xafEntryManager).GetAwaiter().GetResult();
    var semanticTextMemory = GetSemanticTextMemory(store);
    await semanticTextMemory.SaveInformationAsync(currentMemoryChat.Name, id: Guid.NewGuid().ToString(), text: textMemory.Content);
}

Here, the GetSemanticTextMemory method plays a key role:

private static SemanticTextMemory GetSemanticTextMemory(XpoMemoryStore store)
{
    var embeddingModelId = "text-embedding-3-small";
    var getKey = () => Environment.GetEnvironmentVariable("OpenAiTestKey", EnvironmentVariableTarget.Machine);

    var kernel = Kernel.CreateBuilder()
        .AddOpenAIChatCompletion(ChatModelId, getKey.Invoke())
        .AddOpenAITextEmbeddingGeneration(embeddingModelId, getKey.Invoke())
        .Build();

    var embeddingGenerator = new OpenAITextEmbeddingGenerationService(embeddingModelId, getKey.Invoke());
    return new SemanticTextMemory(store, embeddingGenerator);
}

This method sets up an embedding generator used to create semantic memories.

Step 3: Querying Memories

To query the stored memories, I created a non-persistent type that interacts with the chat component:

public interface IMemoryData
{
    IChatCompletionService ChatCompletionService { get; set; }
    SemanticTextMemory SemanticTextMemory { get; set; }
    string CollectionName { get; set; }
    string Prompt { get; set; }
    double MinimumRelevanceScore { get; set; }
}

This interface provides the necessary services to interact with the chat component, including ChatCompletionService and SemanticTextMemory.

Step 4: Handling Messages

Lastly, we handle message-sent callbacks, as explained in this article:

async Task MessageSent(MessageSentEventArgs args)
{
    ChatHistory.AddUserMessage(args.Content);

    var answers = Value.SemanticTextMemory.SearchAsync(
        collection: Value.CollectionName,
        query: args.Content,
        limit: 1,
        minRelevanceScore: Value.MinimumRelevanceScore,
        withEmbeddings: true
    );

    string answerValue = "No answer";
    await foreach (var answer in answers)
    {
        answerValue = answer.Metadata.Text;
    }

    string messageContent = answerValue == "No answer"
        ? "There are no memories containing the requested information."
        : await Value.ChatCompletionService.GetChatMessageContentAsync($"You are an assistant queried for information. Use this data: {answerValue} to answer the question: {args.Content}.");

    ChatHistory.AddAssistantMessage(messageContent);
    args.SendMessage(new Message(MessageRole.Assistant, messageContent));
}

Here, we intercept the message, query the SemanticTextMemory, and use the results to generate an answer with the chat completion service.

This was a long post, but I hope it’s useful for you all. Until next time—XAF OUT!

You can find the full implementation on this repo