Undergraduate

Contextual Robustness and Failure Attribution in Agentic Robot-Assisted PCB Assembly Change-Over

Project PREP0005381 · NIST sponsor Yan Lu

Overview

The work will entail: As part of its Super Intelligence (SI) for Manufacturing project*,
NIST develops the measurement science — methodologies, metrics, datasets, tools,
and benchmark studies — needed to enable trustworthy SI and reliable human-SI
teaming across manufacturing system design, operation, and automation. A central
research thrust concerns the integration and deployment of SI-enabled systems,
supported by an integration test bench, benchmark studies, and digital twin-enabled de-
risking of manufacturing SI. The Research Associate will investigate the contextual
robustness of agentic SI orchestrating manufacturing change-overs, specifically in high-
mix, low-volume printed circuit board (PCB) assembly involving robot-assisted pick-and-
insert and selective soldering. The work will involve developing simulation environments
utilizing the Model Context Protocol (MCP) to interface AI agents with manufacturing
processes. The associate will design and implement automated test harnesses to inject
context perturbations (e.g., sensor noise, semantic variations, conflicting operational
objectives) to evaluate agent safety boundaries and develop black-box causal tracing
frameworks to attribute failures across perception, knowledge retrieval, reasoning, and
tool parameterization layers. The research directly contributes to developing metrics,
datasets, and benchmark studies for trustworthy industrial SI.

Qualifications

  • Working toward a bachelor’s degree in Computer Science, Systems Engineering, Industrial Engineering, Data Science, Quantitative Finance, or a related field. Senior undergraduate students capable of committing 20 hours per week are encouraged to apply.
  • Programming experience with Python, API integration, and orchestration frameworks (e.g., LangGraph).
  • Familiarity with Large Language Models (LLMs), Agentic-AI methods, Retrieval-Augmented Generation (RAG), and the Model Context Protocol (MCP).
  • Experience or strong interest in testing, evaluating, and developing causal attribution methods for SI systems (e.g., stress testing, counterfactual analysis).
  • Familiarity with digital twin concepts, simulation tools, or manufacturing systems (e.g., PCB assembly, robotics, process control) is desirable.
  • Strong analytical and problem-solving skills with the ability to develop software prototypes and benchmark harnesses.
  • Strong oral and written communication skills.

Research Proposal

Key responsibilities will include but are not limited to:

  • Developing a standardized Model Context Protocol (MCP) tool environment that interfaces agentic orchestration frameworks (e.g., LangGraph) with simulated PCB assembly cells encompassing pick-and-insert and selective soldering operations.
  • Designing and implementing automated perturbation suites to systematically introduce contextual variability (syntactic/semantic shifts in BOMs and prompts), sensor and metrology degradation, and conflicting operational objectives into the agent's operational environment.
  • Executing batch evaluation trials and benchmark studies to measure the agent's robustness, safety limit adherence, and failure distributions under nominal versus perturbed change-over scenarios.
  • Developing and deploying black-box counterfactual tracing frameworks to automatically classify and localize SI failure modes (e.g., perception errors, RAG failures, reasoning violations, tool parameterization errors).
  • Conducting ablation studies to quantify how manufacturing context (e.g., metrology tolerances, ontologies like Manufacturing Operation Management Ontology) impacts agent decision grounding.
  • Presenting results at internal meetings and occasional meetings with external stakeholders, and contributing to technical reports, datasets, open-source code repositories, and relevant standards-development efforts.