by Joche Ojeda | Feb 23, 2025 | A.I
I’ve been thinking about this topic for a while and have collected numerous notes and ideas about how to present abstractions that allow large language models (LLMs) to interact with various systems – whether that’s your database, operating system, word documents, or other applications.
Before diving deeper, let’s review some fundamental concepts:
Key Concepts
First, let’s talk about APIs (Application Programming Interface). In simple terms, an API is a way to expose methods, functions, and procedures from your application, independent of the programming language being used.
Next is the REST API concept, which is a method of exposing your API using HTTP verbs. As IT professionals, we hear these terms – HTTP, REST, API – almost daily, but we might not fully grasp their core concepts. Let me explain how they relate to software automation using AI.
HTTP (Hypertext Transfer Protocol) is fundamentally a way for two applications to communicate using text. This is its beauty – text serves as the basic layer of understanding between systems, meaning almost any system or programming language can produce a client or server that can interact via HTTP.
REST (Representational State Transfer) is a methodology for systems to communicate and either change or read the state of another system.
Levels of System Interaction
When implementing LLMs for system automation, we first need to determine our desired level of interaction. Here are several approaches:
- Human-like Interaction: An LLM can interact with your operating system using mouse and keyboard inputs, effectively mimicking human behavior.
- REST API Integration: Your application can communicate using HTTP verbs and the REST protocol.
- SDK Implementation: You can create a software development kit that describes your application’s functionality and expose this to the LLM.
The connection method will vary depending on your chosen technology. For instance:
- Microsoft Semantic Kernel allows you to create plugins that interact with your system through REST API, database, or SDK.
- Microsoft AI extensions require you to decide on your preferred interaction level before implementation.
- The Model Context Protocol is a newer approach that enables application exposure for LLM agents, with Claude from Anthropic being a notable example.
Implementation Considerations
When automating your system, you need to consider:
- Available Integration Options: Not all systems provide an SDK or API, which can limit automation possibilities.
- Interaction Protocol Choice: You’ll need to decide between REST API, HTTP, or Model Context Protocol.
This overview should help you understand the various levels of resolution needed to automate your application. What’s your preferred method for integrating LLMs with your applications? I’d love to hear your thoughts and experiences.
by Joche Ojeda | Sep 4, 2024 | A.I, Semantic Kernel
In our continued exploration of Semantic Kernel, we shift focus towards its memory capabilities, specifically diving into the Microsoft.SemanticKernel.Memory Namespace. Here, we’ll discuss the critical components that allow for efficient memory management and how you can integrate your own custom memory stores for AI applications. One of the standout implementations within this namespace is the VolatileMemoryStore, but equally important is understanding how any class that implements IMemoryStore can serve as a backend for SemanticTextMemory.
What Is “Memory” in Semantic Kernel?
Before we dive into the technical details, let’s clarify what we mean by “memory” within the Semantic Kernel framework. When we refer to “memory,” we are not talking about RAM or the typical computer memory used to store data for a short period of time. In the context of Semantic Kernel, memory refers to a record or a unit of information, much like a piece of information you might recall from personal experience. This could be an entry stored in a memory store, which later can be retrieved, searched, or modified.
The Microsoft.SemanticKernel.Memory Namespace
This namespace contains several important classes that serve as the foundation for memory management in the kernel. These include but are not limited to:
- MemoryRecord: The primary schema for memory storage.
- MemoryRecordMetadata: Handles metadata associated with a memory entry.
- SemanticTextMemory: Implements methods to save, retrieve, and search for text-based information in a memory store.
- VolatileMemoryStore: A simple, in-memory implementation of a memory store, useful for short-term storage during runtime.
- IMemoryStore: The interface that defines how a memory store should behave. Any class that implements this interface can act as a memory backend for SemanticTextMemory.
VolatileMemoryStore: A Simple Example
The VolatileMemoryStore is an example of a non-persistent memory store. It operates in-memory and is well-suited for temporary storage needs that do not require long-term persistence. The class implements IMemoryStore, which means that it provides all the essential methods for storing and retrieving records in the memory.
Some of its key methods include:
- CreateCollectionAsync: Used to create a collection of records.
- DeleteCollectionAsync: Deletes a collection from the memory.
- UpsertAsync: Inserts or updates a memory record.
- GetAsync: Retrieves a memory record by ID.
- GetNearestMatchesAsync: Finds memory records that are semantically closest to the input.
Given that the VolatileMemoryStore does not offer persistence, it is primarily suited for short-lived applications. However, because it implements IMemoryStore, it can be replaced with a more persistent memory backend if required.
IMemoryStore: The Key to Custom Memory Implementations
The power of the Semantic Kernel lies in its flexibility. As long as a class implements the IMemoryStore interface, it can be used as a memory backend for the kernel’s memory management. This means that you are not limited to using the VolatileMemoryStore. Instead, you can develop your own custom memory stores by following the pattern established by this class.
For instance, if you need to store memory records in a database, or in a distributed cloud storage solution, you can implement the IMemoryStore interface and define how records should be inserted, retrieved, and managed within your custom store. Once implemented, this custom memory store can be used by the SemanticTextMemory class to manage text-based memories.
SemanticTextMemory: Bringing It All Together
At the core of managing memory in the Semantic Kernel is the SemanticTextMemory class. This class interacts with your memory store to save, retrieve, and search for text-based memory records. Whether you’re using the VolatileMemoryStore or a custom implementation of IMemoryStore, the SemanticTextMemory class serves as the interface that facilitates text-based memory operations.
Key methods include:
- SaveInformationAsync: Saves information into the memory store, maintaining a copy of the original data.
- SearchAsync: Allows for searching the memory based on specific criteria, such as finding semantically similar records.
- GetCollectionsAsync: Retrieves available collections of memories from the memory store.
Why Flexible Memory Stores Matter
The flexibility provided by the IMemoryStore interface is critical because it allows developers to integrate the Semantic Kernel into a wide variety of systems, from small-scale in-memory applications to enterprise-grade solutions that require persistent storage across distributed environments. Whether you’re building an AI agent that needs to “remember” user inputs between sessions or a bot that needs to recall relevant details during an interaction, the memory system in the Semantic Kernel is built to scale with your needs.
Final Thoughts
In the world of AI, memory is a crucial component that makes AI agents more intelligent and capable of handling complex interactions. With the Microsoft.SemanticKernel.Memory Namespace, developers have a flexible, scalable solution for memory management. Whether you’re working with the in-memory VolatileMemoryStore or designing your custom memory backend, the IMemoryStore interface ensures seamless integration with the Semantic Kernel’s memory system.
By understanding the relationship between IMemoryStore, VolatileMemoryStore, and SemanticTextMemory, you can harness the full potential of the Semantic Kernel to create more sophisticated AI-driven applications.
Memory Types in Semantic Kernel
by Joche Ojeda | Sep 4, 2024 | A.I, Semantic Kernel
In the world of AI and large language models (LLMs), understanding how to manage memory is crucial for creating applications that feel responsive and intelligent. Many developers are turning to Semantic Kernel, a lightweight and open-source development kit, to integrate these capabilities into their applications. For those already familiar with Semantic Kernel, let’s dive into how memory functions within this framework, especially when interacting with LLMs via chat completions.
Chat Completions: The Most Common Interaction with LLMs
When it comes to interacting with LLMs, one of the most intuitive and widely used methods is through chat completions. This allows developers to simulate a conversation between a user and an AI agent, facilitating various use cases like building chatbots, automating business processes, or even generating code.
In Semantic Kernel, chat completions are implemented through models from popular providers like OpenAI, Google, and others. These models enable developers to manage the flow of conversation seamlessly. While using chat completions, one key aspect to keep in mind is how the conversation history is stored and managed.
Temporary Memory: ChatHistory and Kernel String Arguments
Within the Semantic Kernel framework, the memory that a chat completion model uses is managed by the ChatHistory object. This object stores the conversation history temporarily, meaning it captures the back-and-forth between the user and the model during an active session. Alternatively, you can use a string argument passed to the kernel, which contains context information for the conversation. However, like the ChatHistory, this method is also not persistent.
Once the host class is disposed of, all stored context and memory from both the ChatHistory object and the string argument are lost. This transient nature of memory means that these methods are useful only for short-term interactions and are destroyed after the session ends.
What’s Next? Exploring Long-Term Memory Options
In this article, we’ve discussed how Semantic Kernel manages short-term memory with ChatHistory and kernel string arguments. However, for more complex applications that require retaining memory over longer periods—think customer support agents or business process automation—temporary memory might not be sufficient. In the next article, we’ll explore the options available for implementing long-term memory within Semantic Kernel, providing insights on how to make your AI applications even more powerful and context-aware.
Stay tuned for the deep dive into long-term memory solutions!
by Joche Ojeda | Apr 29, 2024 | Semantic Kernel
Welcome to the fascinating world of artificial intelligence (AI)! You’ve probably heard about AI’s incredible potential to transform our lives, from smart assistants in our homes to self-driving cars. But have you ever wondered how all these intelligent systems communicate and work together? That’s where something called “Semantic Kernel Connectors” comes in.
Imagine you’re organizing a big family reunion. To make it a success, you need to coordinate with various family members, each handling different tasks. Similarly, in the world of AI, different parts need to communicate and work in harmony. Semantic Kernel Connectors are like the family members who help pass messages and coordinate tasks to ensure everything runs smoothly.
These connectors are a part of a larger system known as the Semantic Kernel framework. They act as messengers, allowing different AI models and external systems, like databases, to talk to each other. This communication is crucial because it lets AI systems perform complex tasks, such as sending emails or updating records, just like a helpful assistant.
For developers, these connectors are a dream come true. They make it easier to create AI applications that can understand and respond to us just like a human would. With these tools, developers can build more sophisticated AI agents that can automate tasks and even learn from their interactions, here is a list of what you get out of the box.
Core Plugins Overview
- ConversationSummaryPlugin: Summarizes conversations to provide quick insights.
- FileIOPlugin: Reads and writes to the filesystem, essential for managing data.
- HttpPlugin: Calls APIs, which allows the AI to interact with web services.
- MathPlugin: Performs mathematical operations, handy for calculations.
- TextMemoryPlugin: Stores and retrieves text in memory, useful for recalling information.
- TextPlugin: Manipulates text strings deterministically, great for text processing.
- TimePlugin: Acquires time of day and other temporal information, perfect for time-related tasks.
- WaitPlugin: Pauses execution for a specified amount of time, useful for scheduling.
So, next time you ask your smart device to play your favorite song or remind you of an appointment, remember that there’s a whole network of AI components working together behind the scenes, thanks to Semantic Kernel Connectors. They’re the unsung heroes making our daily interactions with AI seamless and intuitive.
Isn’t it amazing how far technology has come? And the best part is, we’re just getting started. As AI continues to evolve, we can expect even more incredible advancements that will make our lives easier and more connected. So, let’s embrace this journey into the future, hand in hand with AI.
by Joche Ojeda | Apr 28, 2024 | A.I
Introduction to Semantic Kernel
Hey there, fellow curious minds! Let’s talk about something exciting today—Semantic Kernel. But don’t worry, we’ll keep it as approachable as your favorite coffee shop chat.
What Exactly Is Semantic Kernel?
Imagine you’re in a magical workshop, surrounded by tools. Well, Semantic Kernel is like that workshop, but for developers. It’s an open-source Software Development Kit (SDK) that lets you create AI agents. These agents aren’t secret spies; they’re little programs that can answer questions, perform tasks, and generally make your digital life easier.
Here’s the lowdown:
- Open-Source: Think of it as a community project. People from all walks of tech life contribute to it, making it better and more powerful.
- Software Development Kit (SDK): Fancy term, right? But all it means is that it’s a set of tools for building software. Imagine it as your AI Lego set.
- Agents: Nope, not James Bond. These are like your personal AI sidekicks. They’re here to assist you, not save the world (although that would be cool).
A Quick History Lesson
About a year ago, Semantic Kernel stepped onto the stage. Since then, it’s been striding confidently, like a seasoned performer. Here are some backstage highlights:
- GitHub Stardom: On March 17th, 2023, it made its grand entrance on GitHub. And guess what? It got more than 17,000 stars! (Around 18.2. right now) That’s like being the coolest kid in the coding playground.
- Downloads Galore: The C# kernel (don’t worry, we’ll explain what that is) had 1000000+ NuGet downloads. It’s like everyone wanted a piece of the action.
- VS Code Extension: Over 25,000 downloads! Imagine it as a magical wand for your code editor.
And hey, the .Net kernel even threw a party—it reached a 1.0 release! The Python and Java kernels are close behind with their 1.0 Release Candidates. It’s like they’re all graduating from AI university.
Why Should You Care?
Now, here’s the fun part. Why should you, someone with a lifetime of wisdom and curiosity, care about this?
- Microsoft Magic: Semantic Kernel loves hanging out with Microsoft products. It’s like they’re best buddies. So, when you use it, you get to tap into the power of Microsoft’s tech universe. Fancy, right? Learn more
- No Code Rewrite Drama: Imagine you have a favorite recipe (let’s say it’s your grandma’s chocolate chip cookies). Now, imagine you want to share it with everyone. Semantic Kernel lets you do that without rewriting the whole recipe. You just add a sprinkle of AI magic! Check it out
- LangChain vs. Semantic Kernel: These two are like rival chefs. Both want to cook up AI goodness. But while LangChain (built around Python and JavaScript) comes with a full spice rack of tools, Semantic Kernel is more like a secret ingredient. It’s lightweight and includes not just Python but also C#. Plus, it’s like the Assistant API—no need to fuss over memory and context windows. Just cook and serve!
So, my fabulous friend, whether you’re a seasoned developer or just dipping your toes into the AI pool, Semantic Kernel has your back. It’s like having a friendly AI mentor who whispers, “You got this!” And with its growing community and constant updates, Semantic Kernel is leading the way in AI development.
Remember, you don’t need a PhD in computer science to explore this—it’s all about curiosity, creativity, and a dash of Semantic Kernel magic. ?✨
Ready to dive in? Check out the Semantic Kernel GitHub repository for the latest updates
