Evaluating Methods for End-User Creation of Robot Task Plans

Chris Paxton, Felix Jonathan, Andrew Hundt, Bilge Mutlu, Gregory Hager

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Abstract

How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effctive. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as 'pick up component A from the right side of the table.' Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.

Original languageEnglish (US)
Title of host publication2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages6086-6092
Number of pages7
ISBN (Electronic)9781538680940
DOIs
StatePublished - Dec 27 2018
Event2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 - Madrid, Spain
Duration: Oct 1 2018Oct 5 2018

Publication series

NameIEEE International Conference on Intelligent Robots and Systems
ISSN (Print)2153-0858
ISSN (Electronic)2153-0866

Conference

Conference2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018
CountrySpain
CityMadrid
Period10/1/1810/5/18

Fingerprint

Robots

ASJC Scopus subject areas

  • Control and Systems Engineering
  • Software
  • Computer Vision and Pattern Recognition
  • Computer Science Applications

Cite this

Paxton, C., Jonathan, F., Hundt, A., Mutlu, B., & Hager, G. (2018). Evaluating Methods for End-User Creation of Robot Task Plans. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018 (pp. 6086-6092). [8594127] (IEEE International Conference on Intelligent Robots and Systems). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IROS.2018.8594127

Evaluating Methods for End-User Creation of Robot Task Plans. / Paxton, Chris; Jonathan, Felix; Hundt, Andrew; Mutlu, Bilge; Hager, Gregory.

2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. p. 6086-6092 8594127 (IEEE International Conference on Intelligent Robots and Systems).

Research output: Chapter in Book/Report/Conference proceedingConference contribution

Paxton, C, Jonathan, F, Hundt, A, Mutlu, B & Hager, G 2018, Evaluating Methods for End-User Creation of Robot Task Plans. in 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018., 8594127, IEEE International Conference on Intelligent Robots and Systems, Institute of Electrical and Electronics Engineers Inc., pp. 6086-6092, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018, Madrid, Spain, 10/1/18. https://doi.org/10.1109/IROS.2018.8594127
Paxton C, Jonathan F, Hundt A, Mutlu B, Hager G. Evaluating Methods for End-User Creation of Robot Task Plans. In 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. Institute of Electrical and Electronics Engineers Inc. 2018. p. 6086-6092. 8594127. (IEEE International Conference on Intelligent Robots and Systems). https://doi.org/10.1109/IROS.2018.8594127
Paxton, Chris ; Jonathan, Felix ; Hundt, Andrew ; Mutlu, Bilge ; Hager, Gregory. / Evaluating Methods for End-User Creation of Robot Task Plans. 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018. Institute of Electrical and Electronics Engineers Inc., 2018. pp. 6086-6092 (IEEE International Conference on Intelligent Robots and Systems).
@inproceedings{9b33690344694750992a0fc75452b938,
title = "Evaluating Methods for End-User Creation of Robot Task Plans",
abstract = "How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effctive. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as 'pick up component A from the right side of the table.' Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.",
author = "Chris Paxton and Felix Jonathan and Andrew Hundt and Bilge Mutlu and Gregory Hager",
year = "2018",
month = "12",
day = "27",
doi = "10.1109/IROS.2018.8594127",
language = "English (US)",
series = "IEEE International Conference on Intelligent Robots and Systems",
publisher = "Institute of Electrical and Electronics Engineers Inc.",
pages = "6086--6092",
booktitle = "2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018",

}

TY - GEN

T1 - Evaluating Methods for End-User Creation of Robot Task Plans

AU - Paxton, Chris

AU - Jonathan, Felix

AU - Hundt, Andrew

AU - Mutlu, Bilge

AU - Hager, Gregory

PY - 2018/12/27

Y1 - 2018/12/27

N2 - How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effctive. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as 'pick up component A from the right side of the table.' Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.

AB - How can we enable users to create effective, perception-driven task plans for collaborative robots? We conducted a 35-person user study with the Behavior Tree-based CoSTAR system to determine which strategies for end user creation of generalizable robot task plans are most usable and effctive. CoSTAR allows domain experts to author complex, perceptually grounded task plans for collaborative robots. As a part of CoSTAR's wide range of capabilities, it allows users to specify SmartMoves: abstract goals such as 'pick up component A from the right side of the table.' Users were asked to perform pick-and-place assembly tasks with either SmartMoves or one of three simpler baseline versions of CoSTAR. Overall, participants found CoSTAR to be highly usable, with an average System Usability Scale score of 73.4 out of 100. SmartMove also helped users perform tasks faster and more effectively; all SmartMove users completed the first two tasks, while not all users completed the tasks using the other strategies. SmartMove users showed better performance for incorporating perception across all three tasks.

UR - http://www.scopus.com/inward/record.url?scp=85063005868&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85063005868&partnerID=8YFLogxK

U2 - 10.1109/IROS.2018.8594127

DO - 10.1109/IROS.2018.8594127

M3 - Conference contribution

T3 - IEEE International Conference on Intelligent Robots and Systems

SP - 6086

EP - 6092

BT - 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems, IROS 2018

PB - Institute of Electrical and Electronics Engineers Inc.

ER -