Most organisations already have valuable business data sitting inside their CRM. Customer records, contacts, opportunities, cases, notes and follow-ups are usually all there, but finding the right information quickly is not always easy.

Traditional CRM systems are excellent at storing structured information, but they are not always natural to interact with. A user may know what they want to ask, but not which module to open, which filter to apply, or which field contains the answer.

This is where AI becomes interesting:

“Show recent accounts.”

“Retrieve Chloe Martin contact.”

“Show opportunities greater than 1000.”

“List first 10 accounts.”

From Business Challenge to AI Roadmap

When I was approached by the customer, the conversation was never about deploying an AI model or choosing a Large Language Model. The customer already had a mature CRM system containing valuable business information, but employees were struggling to retrieve that information quickly. Finding the right customer, opportunity, contact or support record often meant navigating multiple modules, filters and screens. The challenge was not the lack of data, it was the difficulty of accessing it in a natural and efficient way.

Rather than jumping directly into development, we worked together to build a roadmap focused on business outcomes. Every technical decision was driven by a simple question: How will this make life easier for the people using the CRM every day?

Our approach followed several discovery stages before a single line of code was written:

• Business Discovery: Understand how teams currently work, identify pain points, and define measurable business outcomes.
• Business Challenges: Document where users lose time, where information becomes difficult to locate, and which repetitive tasks reduce productivity.
• Desired Outcomes: Define what success looks like from the customer’s perspective, focusing on faster access to information rather than replacing existing business systems.
• Technical Discovery: Review the CRM architecture, APIs, authentication methods, security requirements and available integration points.
• Data Readiness Assessment: Evaluate the quality, consistency and accessibility of CRM data to ensure an AI assistant could retrieve reliable information.
• Technology Selection: Choose the right architecture and components, balancing capability, privacy, operational simplicity and long-term maintainability.

Only after completing these stages did we move into building the first Minimum Viable Product (MVP). This ensured we were solving the right business problem instead of simply implementing another AI project. The result was a practical conversational assistant that allowed users to interact with their CRM using natural language while keeping all business data securely within the customer’s environment.

The goal was not to replace the CRM interface. The goal was to create a conversational layer on top of the CRM that can help users retrieve and understand information faster.

Why I Chose a Local AI Approach

One of the main design decisions was to use a local LLM instead of sending CRM data directly to a public cloud AI service.

CRM data can be sensitive. It may include customer names, phone numbers, email addresses, sales opportunities, support cases and internal notes. For many businesses, especially those dealing with regulated or commercially sensitive information, sending this data outside the organisation is not always acceptable.

Following the discovery and design phases, we selected Ollama as the local LLM runtime because it best aligned with the customer’s security, privacy, and operational requirements. Keeping the model inside the customer’s environment meant sensitive CRM data remained on-premises while giving the business full control over performance, governance, and future scalability.

This approach also makes the solution more cost predictable. Instead of every question becoming an external API call, the model runs entirely within the customer’s environment, allowing the same AI platform to support multiple internal business use cases while maintaining complete control over security, governance, and operational costs.

The decision to use a local AI platform was based on several business and technical considerations:

• Complete Control: The organisation retains full control over model selection, updates, security policies, backup, monitoring, and lifecycle management.
• Data Sovereignty: The customer manages sensitive customer information, making it essential that CRM data never leaves their private data centre.
• Privacy and Compliance: Customer records remain under the organisation’s full control, reducing regulatory and compliance concerns.
• Predictable Operating Costs: There are no per-prompt or per-token charges. Once deployed, ongoing costs are largely limited to electricity and infrastructure maintenance.
• Existing Infrastructure Investment: The customer already owned and operated a private data centre, meaning there was no requirement to invest in new servers. The only additional hardware consideration was a suitable GPU, which represents a relatively small investment compared to long-term cloud AI subscription costs.
• Leverage Existing Skills: The customer’s infrastructure team already had the skills to manage Linux, Docker, networking, and virtualisation, allowing the AI platform to be supported using existing operational processes.
• Performance and Low Latency: Processing requests locally provides faster response times and eliminates dependency on Internet connectivity for AI queries.
• Scalable Internal Platform: The same AI infrastructure can later support additional business applications such as document search, internal knowledge assistants, help desk automation, and workflow assistants without additional AI subscription costs.

The Architecture

The solution was built using a simple container-based architecture. The main components were:

• CRM as the source CRM system
• FastAPI as the controlled API layer
• Ollama as the local LLM engine
• Nginx as the Web UI and reverse proxy
• Docker Compose for deployment and networking

The high-level flow looks like this:

This separation is important. The AI model should not be responsible for blindly searching the CRM. The backend should retrieve the correct data first, and the AI should only help explain, summarise or present it.

Why FastAPI Was Important

FastAPI became the control layer between the user, CRM and the local LLM.

This layer is where the real enterprise value sits.

Instead of giving the AI assistant direct access to everything, the API controls:

• Which CRM modules are allowed
• Whether the assistant can read, write or delete
• How many records can be retrieved
• Which fields are returned
• When Ollama should be used
• How results are logged
• What guardrails are applied

For the first version, I intentionally kept the assistant read-only. It can query records, but it cannot create, update or delete CRM data.

This is an important design principle for AI systems connected to business applications. Start with visibility. Add action only when the controls, approvals and audit trail are ready.

Connecting to SuiteCRM

The SuiteCRM integration used OAuth2 and the SuiteCRM API. The assistant connects using a dedicated API user rather than an administrator account. This is important because the AI assistant should only have the permissions it needs.

The API layer can retrieve records from modules such as:

• Accounts
• Contacts
• Leads
• Opportunities
• Cases
• Notes
• Tasks

One of the lessons from the project was that natural-language search is not the same as API filtering.For example, a user may ask:

“Retrieve Chloe Martin contact.”

A CRM API may not understand that as a single filter unless the backend separates first name and last name, searches the right fields, and handles fallback matching.

The same applies to questions like:

“Show opportunities greater than 1000.”

That requires the backend to detect the number, understand the comparison, retrieve opportunities and filter by amount.

This is why deterministic backend logic is important. The LLM should not guess CRM records. The API should retrieve them.

The Web Interface

I also built a simple full-page web interface for the assistant.

The interface includes:

A dashboard-style layout.

A main chat area.

Quick action buttons.

A side panel explaining what the assistant can do.

A read-only mode indicator.

A clean full-screen layout that avoids unnecessary page scrolling.

The web interface communicates with the backend using an Nginx reverse proxy. This avoids exposing the internal FastAPI container directly to the browser and keeps the architecture cleaner.

The browser sends requests to the same web endpoint, and Nginx forwards /api requests internally to FastAPI.

This made the application easier to access and more suitable for future publishing behind a proper domain and SSL certificate.

What Worked Well

The project successfully demonstrated that a local AI assistant can be connected to a CRM system using a controlled backend layer.

Some of the working examples included:

• Listing accounts.
• Retrieving CRM records.
• Searching contacts.
• Filtering opportunities by amount.
• Using the local LLM to explain CRM results.

Running the solution with Docker containers. Keeping CRM, Ollama, FastAPI and the web interface separated but connected.

It also demonstrated an important pattern for enterprise AI: do not let the LLM do everything.

Use normal application logic for data retrieval, filtering and permissions. Use the LLM where it adds value: summarisation, explanation, recommendations and natural-language interaction.

What Was Challenging

The most challenging part was not connecting to the LLM. It was making CRM search reliable.

A CRM is structured data. User questions are unstructured. The difficult part is translating a human question into the correct CRM operation.

For example:

“List first 10 accounts” should not go to the LLM. It should return directly from CRM.

“Retrieve Chloe Martin contact” should search contacts by name.

“Opportunities greater than 1000” should detect a numeric comparison and filter the opportunity amount field.

“Summarise this customer” is where the LLM becomes useful.

This project made one thing very clear: a good AI business application is not just a prompt. It is an application architecture.

The LLM is only one component.

Guardrails and Safe Design

Because this assistant connects to real CRM data, guardrails are essential. In this version, the assistant was designed with several safety controls:

• Read-only mode.

• Allowed CRM modules only
• No delete actions
• No write actions
• Controlled record limits
• Backend filtering before AI summarisation
• Dedicated API credentials
• Audit logging
• System prompt instructions to avoid inventing records

This is important because business users need trust. If the assistant does not find a record, it should say that clearly. It should not invent customer names, phone numbers, opportunities or notes.

Where This Can Go Next

This project is a strong foundation for a more advanced CRM AI assistant. Possible next steps include:

• Better CRM search tools
• Customer summaries
• Opportunity risk scoring
• Follow-up recommendations
• Case analysis
• Automatic note drafting
• CRM record update workflows with approval
• Role-based access control
• Integration with email and calendar
• Retrieval across documents and CRM together

The most important next step is to separate the assistant into clear internal tools, such as:

• Search accounts
• Search contacts
• List opportunities
• Filter opportunities by amount
• Summarise customer history
• Recommend next action

This would make the assistant more reliable and easier to extend.

Final Thoughts

This project started as a simple idea: can we connect a local AI model to CRM and make CRM data easier to interact with?

The answer is yes, but the real lesson is deeper.

The value of AI in business systems does not come from simply connecting a chatbot to a database. The value comes from designing the right control layer between the user, the business application and the model.

For me, this project demonstrates the kind of practical AI architecture that businesses need: private, controlled, useful and connected to real operational systems.

At CloudOasis, this is the type of AI work I am focused on: helping businesses move beyond AI demos and build practical AI solutions that connect securely to their own systems, data and workflows.