A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection
學年 115
學期 1
出版（發表）日期 2026-11-05
作品名稱 A Two-Stage Generate-Then-Reason Vision–Language Framework for Task-Conditioned Grasp Pose Selection
作品名稱（其他語言）
著者 Bo-Kai Peng; Shao-Kang Huang; Chen-Chien Hsu; Wei-Yen Wang
單位
出版者
著錄名稱、卷期、頁數 Electronics 2026 15(21), p. 5137
摘要 Task-conditioned robotic manipulation requires grasps that are geometrically plausible and compatible with downstream function. Although vision–language models (VLMs) have shown promise for semantic reasoning in robotic manipulation, VLM-selected grasps may not consistently satisfy both requirements. To address this issue, we propose a two-stage generate-then-reason framework for task-conditioned 6-DoF grasp selection. In Stage 1, a VLM localizes task-relevant object regions, followed by segmentation and GraspNet-based generation of plausible grasp candidates. In Stage 2, another VLM ranks grounded grasp candidates according to their compatibility with the specified downstream task. The evaluation in this journal includes a 60-trial full-system physical study and four controlled replay experiments that examine Stage 1 localization, Stage 2 cross-model ranking, repeated-inference reliability, and cross-stage candidate-pool sensitivity. The physical system achieved an overall task success rate of 63.3%. In the replay analysis, Gemini 3 Flash Preview achieved the highest Stage 1 localization accuracy of 95.0% and the lowest median Stage 1 latency of 11.09 s. For Stage 2 ranking, GPT-5.6 Sol achieved the highest Top-1 task-valid rate of 92.9% on identical frozen candidate pools and showed the strongest repeated-inference behavior among the evaluated critics. The cross-stage sensitivity experiment yielded task-valid selection rates ranging from 70.8% to 87.5% across six model combinations, indicating sensitivity of downstream selection to upstream candidate evidence within the evaluated conditions. These results indicate that physical task completion and stage-specific VLM behavior capture different aspects of system performance and should be evaluated separately when assessing VLM-based grasp-selection systems.
關鍵字 task-conditioned grasp pose selection; vision–-language model; cross-model sensitivity; semantic isolation; 6-DoF grasping; offline replay; structured reasoning; robotic manipulation
語言 zh_TW
ISSN 2079-9292
期刊性質 國內
收錄於 SCI
產學合作
通訊作者 Shao-Kang Huang
審稿制度 否
國別 TWN
公開徵稿
出版型式 ,電子版
相關連結

機構典藏連結 ( http://tkuir.lib.tku.edu.tw:8080/dspace/handle/987654321/130050 )