Integrating ChatGPT, Claude or another large language model into your product means calling a provider’s API from your own backend and designing everything around that call: which model, what context it gets, what shape the answer takes, and what happens when it is slow, wrong or expensive. We build these integrations for SaaS products, internal tools and customer-facing assistants. We are an independent engineering team and not a reseller or partner of any model provider. For the wider practice, see AI development.
Which model should you use?
We choose per task, not per brand. OpenAI, Anthropic, Google and open-weight models each have different strengths, prices and data terms, and those change often. Our process:
- Collect real inputs from your product with the answers you would accept. This is your evaluation set.
- Run two or three candidate models against it, including a smaller, cheaper one.
- Compare quality, latency and cost per request.
- Put the winner behind an internal interface so changing the model later is a configuration change, not a rewrite.
That interface also lets a cheaper model handle easy requests while a stronger one handles hard ones.
How does the model get your data?
A model only knows what it was trained on and what you put in the prompt. For anything about your products, customers or documents, we add retrieval: find the relevant passages first, then pass them in with the question. Retrieval gives current answers, lets you show sources, and avoids retraining. Our RAG guide explains the pieces, and the RAG vs fine-tuning comparison covers when changing the model itself makes sense.
How do you get answers your code can use?
Free text is hard to build on. Where the next step in your system needs data, we ask the model for structured output that matches a schema, validate it in code, and retry or fall back when it does not conform. Tool calling follows the same idea: the model asks for a function by name, your code checks the arguments and runs it. Validation on our side is non-negotiable, because a model can return something that looks right and is not.
How do we control cost and latency?
Both come from the same few levers:
- Prompt size: Send fewer, better passages instead of whole documents.
- Model tiering: Use a small model where it scores well enough on your evaluation set.
- Caching: Reuse results for repeated questions and use the provider’s prompt caching where it exists.
- Streaming: Show the answer as it arrives so slow responses feel faster.
- Limits: Per-user and per-request caps, timeouts and retries with backoff.
- Tracking: Log tokens and cost per feature so the invoice never surprises you.
What about privacy and data retention?
Before any customer data reaches a provider, we read the provider’s current data terms and set the account accordingly. Things we check and document with you:
- Whether API inputs are used for training, and how to turn that off.
- Retention period for requests and any zero-retention option for your plan.
- Region and hosting options, and whether a private cloud deployment of the model is available.
- What we strip before sending: names, identifiers, secrets.
- Access control, so a user only gets passages they are allowed to see.
Settings and terms vary by provider and plan, and we confirm them on the provider’s own documentation for your account rather than assuming. Where data cannot leave your environment, we use a privately hosted model or run the pipeline inside your cloud. One RAG project of ours for a US-based Series-B legal-research SaaS ran entirely inside the client’s own cloud environment; see the case study.
What does a typical integration involve?
A first usable version is a narrow feature: summarising a record, answering questions over help articles, drafting a reply, classifying tickets. The steps are the evaluation set, a prototype on real data, a guarded release to a small group, then monitoring of quality and cost. The same flow applies to larger builds described in AI development services. If the feature needs to take actions in other systems, that is an agent question. If you want engineers on your team to do it, see hire AI developers, or contact us with the feature you have in mind.