Hiring for AI is unusually hard right now because the job title means five different things and everyone's CV mentions the same tools. This guide is written from the other side of the table: we are an AI development company, and what follows is what we would tell a friend hiring for their first AI project — including the parts that are inconvenient for us.
Short answer: Hire for shipped production systems, not model theory. The skill that decides whether your project works is integration and evaluation engineering, not knowing how a transformer works. Ask for one thing running in production and how they measured that it was correct — most candidates cannot answer the second half.
1. Work Out Which Role You Actually Need
These titles overlap in job ads and almost never overlap in practice. Hiring the wrong one wastes a quarter.
| Role | What They Actually Do | Hire When |
|---|---|---|
| AI / LLM engineer | Connects frontier models to your data and systems | Most business AI projects |
| ML engineer | Trains and deploys custom models on your data | Forecasting, scoring, vision tasks |
| Data engineer | Builds the pipelines that feed everything else | Your data is scattered or messy |
| ML researcher | Novel architectures and training methods | You are building models, not products |
| MLOps / platform engineer | Keeps deployed systems reliable and observable | You already run AI in production |
For a first project — a support assistant, an internal knowledge base, a workflow agent — you almost certainly want an AI/LLM engineer, plus data engineering help if your records are in poor shape. You very likely do not want a researcher, and you should be slightly suspicious if someone tries to sell you one.
2. The Skills That Actually Predict Success
Ranked by how often they are the difference between a system that ships and one that quietly gets switched off:
- Integration engineering. Authentication, rate limits, retries, webhooks, dealing with an API that lies about its own schema. Unglamorous, and the majority of the work.
- Evaluation discipline. Can they build a test set of real questions with graded answers and report accuracy honestly, including where it fails? This single habit separates professionals from demo-builders.
- Retrieval design. Chunking, hybrid search, re-ranking, and knowing why stuffing twenty chunks into a prompt makes answers worse and more expensive. See our production RAG pipelines guide for what competence looks like here.
- Failure-mode thinking. What happens when the model does not know, when the API is down, when a user asks something out of scope, when retrieval returns nothing relevant.
- Cost awareness. An engineer who cannot estimate your monthly token bill will build something you cannot afford to run at volume.
- Plain-language communication. They will need to explain a probabilistic system's limits to non-technical stakeholders. If they cannot explain it to you, they cannot explain it to your board.
Notice what is not on that list: fine-tuning, model architecture, and framework name-dropping. Those matter occasionally. The six above matter every time.
3. Freelancer vs Agency vs In-House
| Freelancer | Agency | In-House | |
|---|---|---|---|
| Typical cost | $50–$250/hr | Fixed project price | $130K–$220K/yr |
| Time to start | Days | 1–3 weeks | 2–4 months to hire |
| Skill coverage | One specialty | Full stack of skills | Grows over time |
| Who carries delivery risk | You | Vendor (if fixed-scope) | You |
| Needs technical management | Yes, significantly | Minimal | Yes |
| Best for | One defined component | First production system | AI as core product |
A sequence that works well: use an agency to build and hand over version one, then hire in-house to own and extend it. You get a working system without a four-month hiring gap, and your eventual hire inherits documented code rather than a blank repository and a mandate.
Where a freelancer beats us: if you already have a strong engineering lead and need one specific piece built — an embedding pipeline, a single integration — a good freelancer is cheaper and just as effective. Agencies earn their premium on multi-skill, fixed-scope delivery, not on writing better Python.
4. Eight Interview Questions That Reveal Competence
- "Show me something in production that real users depend on." Not a demo, not a notebook. Then ask what broke after launch.
- "How did you know it was working?" You want to hear about a test set, graded answers, and a measured accuracy number. Vague answers here predict vague results.
- "Tell me about an AI project that failed." Everyone experienced has one. A candidate with no failures has either not shipped or is not being straight with you.
- "What happens when your system doesn't know the answer?" The right answer involves saying so, citing the nearest source, or escalating to a human — never guessing.
- "Estimate the monthly running cost of what I've described." They should ask about query volume and document count before answering. If a number comes out instantly, it is invented.
- "Which parts of my request would you push back on?" Good engineers have opinions about scope. Total agreement is a warning sign, not enthusiasm.
- "What would you need from us to succeed?" Strong candidates ask about data access, subject-matter experts, and who signs off. Weak ones say "just the requirements".
- "Walk me through how you'd handle our permissions." Whether retrieval respects who can see what. If this has never occurred to them, they have not built for a real company.
5. Red Flags — Walk Away
- No code or data ownership in the contract. Non-negotiable. You must own the source, the prompts, and any fine-tuned artefacts.
- Accuracy promised before seeing your data. "95% accurate" quoted from a proposal template is a number with no basis. Nobody can know that before ingestion.
- A quote with no discovery. A same-day fixed price for a system touching your internal tools means either heavy padding or a change-order conversation in month three.
- No evaluation plan. If nobody can tell you how correctness will be measured, nobody will measure it.
- Only the build cost mentioned. Every AI system has a monthly bill. Omitting it is either inexperience or a sales tactic.
- A portfolio of demos. Demos are easy. Ask which of those have been running for six months and who maintains them.
- Buzzword density without specifics. If you cannot get a straight answer about how data flows through the system, there may not be a system.
- Reluctance to talk to a past client. One reference call tells you more than the entire proposal.
The cheapest hiring insurance: pay for a small scoped paid trial — one integration, one document set, two weeks — before signing a large engagement. You will learn more about how someone works from two weeks of real delivery than from any number of interviews, and it costs a fraction of a failed project.
6. What to Have Ready Before You Interview Anyone
The quality of the quotes you receive depends almost entirely on the quality of the brief you give. Before the first call, write down:
- The specific workflow, described as a sequence of steps a person currently performs.
- Which systems the AI must read from and write to, and whether each has an API.
- Where the relevant documents live, roughly how many there are, and what format they are actually in.
- What "working" means, in a number: tickets deflected, minutes saved, error rate reduced.
- Who internally can answer subject-matter questions, and who signs off.
- Your budget range. Withholding it wastes everyone's time and produces worse proposals, not cheaper ones.
Our AI readiness checklist covers this in detail. Clients who arrive with this prepared consistently get fixed-price quotes instead of hourly estimates, because there is less unknown for the vendor to price in.
The Bottom Line
Hire for evidence of shipped, measured systems. Match the engagement model to your situation rather than to fashion — freelancer for a component, agency for a first build, in-house when AI becomes core. And treat any refusal to discuss ownership, accuracy measurement, or monthly running costs as the end of the conversation.
Next, read how to choose an AI development company for the vendor-level checklist, and AI agent development costs or the RAG system pricing guide so you can tell a fair quote from an optimistic one. If you have not yet decided whether to build at all, start with custom AI vs ChatGPT for business.