Model Context Protocol (MCP) is no longer just a buzzword. It is now a well-adopted solution embraced by companies looking to stay ahead of the curve. In fact, our adoption of MCP rendered a significant portion of GenAI software we developed just six months ago obsolete — and that’s no exaggeration.
What is Model Context Protocol (MCP)?
MCP is an open protocol that standardizes how applications provide context to LLMs. MCP provides a standardized way to connect AI models to different data sources and tools. It follows a 4 tier architecture:
- MCP Host: Claude Desktop, Goose Desktop, This is the brain that orchestrates across one or more MCP tools.
- MCP Client: Stub to connect to an MCP Server.
- MCP Server: Provides specific capabilities using tools (resources and prompts). These tools connect with the application to fetch or push data.
- Application: Implements the business logic for the service
Our path to MCP at ComplianceCow
Our GenAI journey began with LangChain, using straightforward LLM calls combined with prompts crafted through various prompt engineering techniques. We later enhanced this setup with LangGraph, incorporating workflows and agents powered by a suite of integrated tools.
We rolled this out to production, gathered feedback, and realized the results fell short of expectations in terms of accuracy.
We researched, analyzed and pushed several incremental updates—such as improving prompts, adding more context, and breaking down complex steps into finer, more manageable ones. Yes, all of these efforts yielded results, but the underlying truth still remained: we could never fully meet the expectations of customers who compare our product to ChatGPT and similar tools.
The primary challenge lay in achieving acceptable fidelity across the LLM steps involved in our AI Assistant use case.
- Generating a concise summary of the user’s chat history
- Determining when to retain or discard prior conversational context
- Extracting the user’s intent correctly from the conversation summary
- Translating the user’s intent into a Cypher query
- Deciding what information from the Cypher query output should be surfaced or withheld in the chat history
💡 The rising buzz around MCP caught our attention, and it emerged as a natural solution to the challenges we faced. Here’s how we reshaped our solution using MCP — the diagram below tells the MCP transition story.
All we needed to do was implement an MCP server equipped with tools and resources capable of fetching data from, and initiating actions within, ComplianceCow. The MCP Host handled the heavy lifting — intelligently deciding which tools from our MCP server to invoke based on the user’s intent. A significant portion of our earlier code — including agents, graph workflows, and prompt logic — was no longer necessary.
👍 We integrated our MCP server with Claude Desktop, and it proved to be a significant success — delivering noticeably higher fidelity in responses. This improvement stemmed primarily from the MCP Host’s ability to better understand context and intelligently select the most appropriate tools for each situation.
The Path Forward
But we can’t stop here — the MCP approach, while powerful, comes with certain limitations that leave us constrained:
🐮 Flexibility: Although MCP supports prompts, it offers limited control over the decisions made by the MCP Host, making it difficult to correct or override tool selection when needed.
🐮 Performance: The layered architecture involving hosts, clients, and servers introduces overhead, which can impact performance to some extent.
🐮 Third-Party Dependency: Customers are required to either build or adopt an MCP Host, adding friction to adoption and increasing reliance on external tools.
The decision on whether to continue with MCP or not will largely depend on how well the MCP community matures and addresses the limitations.
🔹 Want to see if ComplianceCow’s capabilities fit your situation? Book a Demo
🔹 Listen to other conversations with security GRC compliance leads. Click Here