Graphical User Interface

The following provides an overview of the graphical user interface developed for the system, as well as a general flow of actions happening during a game of robotic pong. The user begins with the following display which asks the user which cup they would like the cannon to attempt to shoot the ping pong ball into:

After the user has selected one of the available cups, the required joint angles on the cannon are calculated, and the trajectory to the desired cup is displayed below the current field configuration. The user is then asked if they would like to take the shot, reset the current estimate on the virtual estimation of the location of the robot (in case they have given poor feedback, resulting in a poor estimation), as well as if they would like to select a different cup:

If the user chooses to reset the feedback, the estimated virtual position of the robot is aligned with the origin of the global coordinate frame, and the GUI returns to the ‘Choose Cup View’ as seen in Fig. 1. The user is also returned to the ‘Choose Cup View’ if they select the option to choose a different cup to aim at. If the user chooses to take the shot, the robot heads to the correct joint angles (calculated from inverse kinematics using the estimated virtual robot location, ground truth cup locations, and simple ballistics - see above section on Inverse Kinematics for more information) and launches the ball. The following is then displayed:

The user is now able to select which cup they made, if any. Let us assume that the user chose to shoot at Cup 1, had the robot take the shot, and the ball actually landed in Cup 2. The user is then asked if the would like to provide feedback, while also being given the option to manually input the robot offset:

If the user decides not to provide feedback, they are returned to the ‘Choose Cup View’ with any cups that they made no longer available and crossed out on the GUI. When the user chooses to manually input robot offset, a menu comes up allowing them to type in the virtual location of the robot, after which the user is returned to the ‘Choose Cup View’ as well. However, if the user decides to provide feedback, they are given a display window of the current configuration of the field with a set of cross hairs with which they can click on the location the ball landed. After clicking on the location the ball landed, the user is given the option to continue to the game, end the game, or add a cup back (incase the user accidentally removed a cup or failed to play by house rules):

After the user provides feedback, a new updated estimation of the virtual location of the robot is calculated using the Extended Kalman Filter described in the above section. For our given example, we continue the game and return to the ‘Cup Chooser View’ which appears without Cup 2 as an option since we declared that we hit it earlier: