Classifying a job description into one of the NOC 2021 unit groups is the one place in MaplePath where a wrong answer is plausible and consequential — it decides TEER, and TEER decides program eligibility. So retrieval is measured, not asserted. 9 job descriptions, hand-labelled with their correct NOC code, run through four retrieval methods with identical queries.
Higher NDCG@10 means the correct code sits nearer the top of the list. The shipped method is the last row.
One row per labelled description, from examples_report. NDCG and hit are for the shipped hybrid + rerank method.
| Job title | Case type | NDCG | Outcome |
|---|---|---|---|
| Software Developer | exact_title | 0.865 | Top 10 |
| Software Engineer | close_sibling | 0.824 | Top 10 |
| IT Support Technician | vague_title | 0.694 | Top 10 |
| Bookkeeper | teer_distinction | 0.476 | Top 10 |
| Registered Nurse | exact_title | 0.723 | Top 10 |
| Licensed Practical Nurse | close_sibling | 0.498 | Top 10 |
| Cook | close_sibling | 0.596 | Top 10 |
| Electrician | exact_title | 0.820 | Top 10 |
| Transport Truck Driver | exact_title | 0.558 | Top 10 |
Labels live in data/noc_eval_labels.csv. This report is produced live by the /evaluate endpoint on each visit.