Engineered a AI-powered talent-recommendation system that cut staffing search to minutes開發 GenAI 人才推薦系統，將專案人力搜尋縮短至數分鐘

Situation

At a global consultancy with more than 8,000 consultants worldwide, Engagement Managers routinely spent several days identifying the right people to staff new projects. Skills, project experience, regional knowledge, and availability were fragmented across CVs and internal systems, leaving managers to conduct manual searches and send broad outreach emails.

This created problems at both ends of the staffing process. Some consultants became overloaded while others remained underutilised. In an hourly billing model, these imbalances affected project delivery, workforce utilisation, and commercial performance.

The objective was to create a system that managers could query in natural language—for example, “Who has transaction monitoring experience in APAC?”—and receive a ranked, evidence-backed shortlist within minutes.

Strategy

As the AI engineer on the project, I owned the retrieval design, model experimentation, and evaluation framework. We designed the solution using retrieval-augmented generation (RAG), grounding every recommendation in internal evidence from consultants’ CVs, project histories, and skills data rather than relying on the model’s general knowledge.

Early testing exposed a critical failure mode: the system could recommend a consultant who lacked a required capability simply because a related term appeared elsewhere in their CV. Also, the LLMs could hallucinate the results. This made retrieval relevance, evidence traceability, and the ability to abstain from weak recommendations central design requirements.

Improving retrieval quality.

Consultant CVs varied considerably in structure and level of detail, making fixed-length document chunking unreliable. Important relationships—such as a skill and the project in which it had been applied—could be separated across different chunks.

Working with the data science team, I tested alternative retrieval strategies and moved the system to semantic chunking. This preserved related experience and contextual evidence within the same retrieved passage, improving the quality of the information supplied to the generation model.

I also experimented on the retrieval prompts and recommendation criteria to distinguish direct evidence of a required skill from a superficial keyword match.

Selecting and grounding the generation model.

I evaluated a combination of open-weight and proprietary language models, including Mistral 7B, T5, Llama 3, and GPT-4. The assessment considered groundedness, answer quality, latency, and operating cost rather than model capability in isolation.

Instead of allowing the selected model to return unrestricted prose, I designed the generation layer to produce structured candidate profiles ranked against the staffing requirements. Each recommendation included supporting evidence from the retrieved source material, allowing managers to understand why a candidate had been shortlisted.

I also introduced guardrails for uncertain requests. When the available evidence was insufficient or the staffing criteria were ambiguous, the system asked a clarifying question rather than producing an unsupported recommendation.

Building a systematic evaluation framework.

Manual review was useful during development but could not provide consistent quality assurance at scale. I therefore designed an automated LLM-as-a-judge framework, informed by RAGAS evaluation principles.

The framework scored each response from one to five across three dimensions:

• Completeness: whether the response addressed all material parts of the staffing request
• Relevance: whether the recommended talents matched the stated requirements
• Groundedness: whether each recommendation was supported by retrieved internal evidence

This created a repeatable evaluation process for comparing models and retrieval strategies during development, while also enabling recommendation quality to be monitored after launch.

Result

The system was launched successfully and adopted by managers as part of their regular staffing workflow. It reduced a search-and-outreach process that had previously taken several days to a matter of minutes. Managers also began using the system to identify colleagues for networking and knowledge sharing—an unanticipated use case that revealed a broader visibility problem across the firm. The project therefore delivered more than a faster staffing search. It created a reusable internal discovery capability across an organisation of more than 8,000 consultants.