by Joche Ojeda | Oct 15, 2024 | A.I, Semantic Kernel, XAF, XPO
A few weeks ago, I forked the Semantic Kernel repository to experiment with it. One of my first experiments was to create a memory provider for XPO. The task was not too difficult; basically, I needed to implement the IMemoryStore interface, add some XPO boilerplate code, and just like that, we extended the Semantic Kernel memory store to support 10+ databases. You can check out the code for the XpoMemoryStore here.
My initial goal in creating the XpoMemoryStore was simply to see if XPO would be a good fit for handling embeddings. Spoiler alert: it was! To understand the basic functionality of the plugin, you can take a look at the integration test here.
As you can see, usage is straightforward. You start by connecting to the database that handles embedding collections, and all you need is a valid XPO connection string:
using XpoMemoryStore db = await XpoMemoryStore.ConnectAsync("XPO connection string");
In my original design, everything worked fine, but I faced some challenges when trying to use my new XpoMemoryStore in XAF. Here’s what I encountered:
- The implementation of XpoMemoryStore uses its own data layer, which can lead to issues. This needs to be rewritten to use the same data layer as XAF.
- The XpoEntry implementation cannot be extended. In some use cases, you might want to use a different object to store the embeddings, perhaps one that has an association with another object.
To address these problems, I introduced the IXpoEntryManager interface. The goal of this interface is to handle object creation and queries.
public interface IXpoEntryManager
{
T CreateObject();
public event EventHandler ObjectCreatedEvent;
void Commit();
IQueryable GetQuery(bool inTransaction = true);
void Delete(object instance);
void Dispose();
}
Now, object creation is handled through the CreateObject<T>
method, allowing the underlying implementation to be changed to use a UnitOfWork
or ObjectSpace
. There’s also the ObjectCreatedEvent
event, which lets you access the newly created object in case you need to associate it with another object. Lastly, the GetQuery<T>
method enables redirecting the search for records to a different type.
I’ll keep updating the code as needed. If you’d like to discuss AI, XAF, or .NET, feel free to schedule a meeting: Schedule a Meeting with us.
Until next time, XAF out!
Related Article
https://www.jocheojeda.com/2024/09/04/using-the-imemorystore-interface-and-devexpress-xpo-orm-to-implement-a-custom-memory-store-for-semantic-kernel/
by Joche Ojeda | Sep 4, 2024 | A.I, Semantic Kernel, XPO
In today’s AI-driven world, the ability to quickly and efficiently store, retrieve, and manage data is crucial for developing sophisticated applications. One tool that helps facilitate this is the Semantic Kernel, a lightweight, open-source development kit designed for integrating AI models into C#, Python, or Java applications. It enables rapid enterprise-grade solutions by serving as an effective middleware.
One of the key concepts in Semantic Kernel is memory—a collection of records, each containing a timestamp, metadata, embeddings, and a key. These memory records can be stored in various ways, depending on how you implement the interfaces. This flexibility allows you to define the storage mechanism, which means you can choose any database solution that suits your needs.
In this blog post, we’ll walk through how to use the IMemoryStore interface in Semantic Kernel and implement a custom memory store using DevExpress XPO, an ORM (Object-Relational Mapping) tool that can interact with over 14 database engines with a single codebase.
Why Use DevExpress XPO ORM?
DevExpress XPO is a powerful, free-to-use ORM created by DevExpress that abstracts the complexities of database interactions. It supports a wide range of database engines such as SQL Server, MySQL, SQLite, Oracle, and many others, allowing you to write database-independent code. This is particularly helpful when dealing with a distributed or multi-environment system where different databases might be used.
By using XPO, we can seamlessly create, update, and manage memory records in various databases, making our application more flexible and scalable.
Implementing a Custom Memory Store with DevExpress XPO
To integrate XPO with Semantic Kernel’s memory management, we’ll implement a custom memory store by defining a database entry class and a database interaction class. Then, we’ll complete the process by implementing the IMemoryStore interface.
Step 1: Define a Database Entry Class
Our first step is to create a class that represents the memory record. In this case, we’ll define an XpoDatabaseEntry
class that maps to a database table where memory records are stored.
public class XpoDatabaseEntry : XPLiteObject {
private string _oid;
private string _collection;
private string _timestamp;
private string _embeddingString;
private string _metadataString;
private string _key;
[Key(false)]
public string Oid { get; set; }
public string Key { get; set; }
public string MetadataString { get; set; }
public string EmbeddingString { get; set; }
public string Timestamp { get; set; }
public string Collection { get; set; }
protected override void OnSaving() {
if (this.Session.IsNewObject(this)) {
this.Oid = Guid.NewGuid().ToString();
}
base.OnSaving();
}
}
This class extends XPLiteObject
from the XPO library, which provides methods to manage the record lifecycle within the database.
Step 2: Create a Database Interaction Class
Next, we’ll define an XpoDatabase
class to abstract the interaction with the data store. This class provides methods for creating tables, inserting, updating, and querying records.
internal sealed class XpoDatabase {
public Task CreateTableAsync(IDataLayer conn) {
using (Session session = new(conn)) {
session.UpdateSchema(new[] { typeof(XpoDatabaseEntry).Assembly });
session.CreateObjectTypeRecords(new[] { typeof(XpoDatabaseEntry).Assembly });
}
return Task.CompletedTask;
}
// Other database operations such as CreateCollectionAsync, InsertOrIgnoreAsync, etc.
}
This class acts as a bridge between Semantic Kernel and the database, allowing us to manage memory entries without having to write complex SQL queries.
Step 3: Implement the IMemoryStore Interface
Finally, we implement the IMemoryStore
interface, which is responsible for defining how the memory store behaves. This includes methods like UpsertAsync
, GetAsync
, and DeleteCollectionAsync
.
public class XpoMemoryStore : IMemoryStore, IDisposable {
public static async Task ConnectAsync(string connectionString) {
var memoryStore = new XpoMemoryStore(connectionString);
await memoryStore._dbConnector.CreateTableAsync(memoryStore._dataLayer).ConfigureAwait(false);
return memoryStore;
}
public async Task CreateCollectionAsync(string collectionName) {
await this._dbConnector.CreateCollectionAsync(this._dataLayer, collectionName).ConfigureAwait(false);
}
// Other methods for interacting with memory records
}
The XpoMemoryStore
class takes advantage of XPO’s ORM features, making it easy to create collections, store and retrieve memory records, and perform batch operations. Since Semantic Kernel doesn’t care where memory records are stored as long as the interfaces are correctly implemented, you can now store your memory records in any of the databases supported by XPO.
Advantages of Using XPO with Semantic Kernel
- Database Independence: You can switch between multiple databases without changing your codebase.
- Scalability: XPO’s ability to manage complex relationships and large datasets makes it ideal for enterprise-grade solutions.
- ORM Abstraction: With XPO, you avoid writing SQL queries and focus on high-level operations like creating and updating objects.
Conclusion
In this blog post, we’ve demonstrated how to integrate DevExpress XPO ORM with the Semantic Kernel using the IMemoryStore
interface. This approach allows you to store AI-driven memory records in a wide variety of databases while maintaining a flexible, scalable architecture.
In future posts, we’ll explore specific use cases and how you can leverage this memory store in real-world applications. For the complete implementation, you can check out my GitHub fork.
Stay tuned for more insights and examples!
by Joche Ojeda | Sep 4, 2024 | A.I, Semantic Kernel
In our continued exploration of Semantic Kernel, we shift focus towards its memory capabilities, specifically diving into the Microsoft.SemanticKernel.Memory Namespace. Here, we’ll discuss the critical components that allow for efficient memory management and how you can integrate your own custom memory stores for AI applications. One of the standout implementations within this namespace is the VolatileMemoryStore, but equally important is understanding how any class that implements IMemoryStore can serve as a backend for SemanticTextMemory.
What Is “Memory” in Semantic Kernel?
Before we dive into the technical details, let’s clarify what we mean by “memory” within the Semantic Kernel framework. When we refer to “memory,” we are not talking about RAM or the typical computer memory used to store data for a short period of time. In the context of Semantic Kernel, memory refers to a record or a unit of information, much like a piece of information you might recall from personal experience. This could be an entry stored in a memory store, which later can be retrieved, searched, or modified.
The Microsoft.SemanticKernel.Memory Namespace
This namespace contains several important classes that serve as the foundation for memory management in the kernel. These include but are not limited to:
- MemoryRecord: The primary schema for memory storage.
- MemoryRecordMetadata: Handles metadata associated with a memory entry.
- SemanticTextMemory: Implements methods to save, retrieve, and search for text-based information in a memory store.
- VolatileMemoryStore: A simple, in-memory implementation of a memory store, useful for short-term storage during runtime.
- IMemoryStore: The interface that defines how a memory store should behave. Any class that implements this interface can act as a memory backend for SemanticTextMemory.
VolatileMemoryStore: A Simple Example
The VolatileMemoryStore is an example of a non-persistent memory store. It operates in-memory and is well-suited for temporary storage needs that do not require long-term persistence. The class implements IMemoryStore, which means that it provides all the essential methods for storing and retrieving records in the memory.
Some of its key methods include:
- CreateCollectionAsync: Used to create a collection of records.
- DeleteCollectionAsync: Deletes a collection from the memory.
- UpsertAsync: Inserts or updates a memory record.
- GetAsync: Retrieves a memory record by ID.
- GetNearestMatchesAsync: Finds memory records that are semantically closest to the input.
Given that the VolatileMemoryStore does not offer persistence, it is primarily suited for short-lived applications. However, because it implements IMemoryStore, it can be replaced with a more persistent memory backend if required.
IMemoryStore: The Key to Custom Memory Implementations
The power of the Semantic Kernel lies in its flexibility. As long as a class implements the IMemoryStore interface, it can be used as a memory backend for the kernel’s memory management. This means that you are not limited to using the VolatileMemoryStore. Instead, you can develop your own custom memory stores by following the pattern established by this class.
For instance, if you need to store memory records in a database, or in a distributed cloud storage solution, you can implement the IMemoryStore interface and define how records should be inserted, retrieved, and managed within your custom store. Once implemented, this custom memory store can be used by the SemanticTextMemory class to manage text-based memories.
SemanticTextMemory: Bringing It All Together
At the core of managing memory in the Semantic Kernel is the SemanticTextMemory class. This class interacts with your memory store to save, retrieve, and search for text-based memory records. Whether you’re using the VolatileMemoryStore or a custom implementation of IMemoryStore, the SemanticTextMemory class serves as the interface that facilitates text-based memory operations.
Key methods include:
- SaveInformationAsync: Saves information into the memory store, maintaining a copy of the original data.
- SearchAsync: Allows for searching the memory based on specific criteria, such as finding semantically similar records.
- GetCollectionsAsync: Retrieves available collections of memories from the memory store.
Why Flexible Memory Stores Matter
The flexibility provided by the IMemoryStore interface is critical because it allows developers to integrate the Semantic Kernel into a wide variety of systems, from small-scale in-memory applications to enterprise-grade solutions that require persistent storage across distributed environments. Whether you’re building an AI agent that needs to “remember” user inputs between sessions or a bot that needs to recall relevant details during an interaction, the memory system in the Semantic Kernel is built to scale with your needs.
Final Thoughts
In the world of AI, memory is a crucial component that makes AI agents more intelligent and capable of handling complex interactions. With the Microsoft.SemanticKernel.Memory Namespace, developers have a flexible, scalable solution for memory management. Whether you’re working with the in-memory VolatileMemoryStore or designing your custom memory backend, the IMemoryStore interface ensures seamless integration with the Semantic Kernel’s memory system.
By understanding the relationship between IMemoryStore, VolatileMemoryStore, and SemanticTextMemory, you can harness the full potential of the Semantic Kernel to create more sophisticated AI-driven applications.
Memory Types in Semantic Kernel