 |  |  |
Jupiter Robot Standard Edition — Technical Specifications
Programmable AI Educational Robot Hardware Platform
This project designed and implemented an extensible, easy-to-maintain robot hardware platform. The platform structure diagram is shown below:
Motion Module and Navigation Module
Motion module：The robot platform uses the Kobuki indoor mobile robot base developed by Yujin Robot. The interface control board that comes with the base has 32 built-in sensors, including infrared sensors, bump sensors, drop sensors, cliff sensors, and a gyroscope. The base is driven by two DC motors with encoders, with a maximum running speed of about 50 cm/s and a maximum rotation speed of 160 degrees/s, and can pass over general indoor obstacles no higher than 12 mm. The base can carry a load of 5 kg and can work continuously for more than 3 hours.
Navigation module：This platform uses a 3D stereo vision camera as the navigation sensor, providing the robot with depth information in three-dimensional space. The depth sensing module of the 3D stereo vision camera uses Light Coding technology: an independent infrared laser emission module generates an infrared laser light-coding pattern (IR Light Coding Image), forming infrared speckles invisible to the naked eye that are projected into space to add markers to objects in the space. Because the speckles are highly random, the speckle patterns at any two locations in space are different. After capturing the infrared image in the space, the receiver hands the image to a dedicated image-processing chip for computation to derive the depth map.
Based on the Kobuki indoor mobile robot base and the 3D stereo vision camera, the platform implements a robot navigation and mapping algorithm based on GMapping and an indoor robot localization algorithm based on AMCL. Based on a pre-established indoor environment map and predefined navigation points, the robot can be guided to travel along a specific route. During travel, the robot also avoids obstacles that appear on the planned path in real time, reaching the navigation target point along a re-planned path.
Using the second 3D stereo vision camera mounted on top of the robot, the robot can also implement following and autonomous guidance tasks in known or even unknown environments. Through integration with the underlying motion platform, this project realizes real-time map updates while the robot follows an operator in an unknown environment. This function is of great significance for safe and effective interaction between home service robots and people in home environments.
Voice Synthesis Module
Voice Synthesis Module：The voice synthesis module consists of two parts — sound collection and sound output. To achieve interaction close to natural language, the platform uses a dedicated microphone with a unidirectional cardioid sound collection range. Compared with omnidirectional microphones, a unidirectional microphone can effectively filter out environmental noise from the sides and rear of the robot and more accurately collect the operator’s voice. At the same time, this microphone also has pickup sensitivity adjustment and low-cut filtering functions, enabling it to collect human voice relatively accurately and avoid the influence of other low-frequency noise on speech recognition. For sound output, a small speaker with small volume and high output power is used. The speaker has an output power of 7 W, an impedance of 4 Ω, and a signal-to-noise ratio greater than 75 dB, and can output a natural voice-synthesis effect relatively clearly even in noisy environments.
To improve speech recognition efficiency, the speech recognition system adopts a strategy of the robot prioritizing listening for an activation keyword. Once the activation keyword is recognized, it switches to a dedicated search to recognize the actual command. Once the command is recognized, the robot can switch to a grammar search to recognize confirmation, and then switch back to keyword-listening mode to wait for another command.
For natural language recognition, the robot uses the CMU Pocket Sphinx speech recognition model based on the Sphinxbase library. The speech recognition system uses the latest version, “sphinxbase-5prealpha”, to build the application. GStreamer is used to automatically split the input audio into the speech to be recognized, and provides services to start and stop recognition. The speech recognition system requires specific language model and dictionary files, which can be automatically built from sentence corpora of natural language. For text-to-speech (TTS), the robot uses the CMU Festival system and the ROS Sound_Play package.
To improve speech recognition accuracy in noisy environments, the Sphinxtrain tool is used within the CMU Pocket Sphinx speech recognition model to train sentence recordings in noisy environments. The Sphinxtrain tool uses a large number of the above recordings as a database to extract the sounds of noisy environments. The obtained parameters are then used to replace the original parameters for better speech recognition detection.
Sound source localization: In addition to human-machine voice interaction, the robot also explores the use of the HARK method to determine the likely position of a speaker within the robot’s virtual predicted region. This method will help the robot determine the speaker’s direction after hearing specific speech in open public places such as restaurants, shopping malls, and airports.
Image Processing Module
Image Processing Module：Image processing needs to realize real-time classification and recognition of targets, which requires the image sensor to have good dynamic performance and high image quality. At the same time, in order to cooperate with the robotic arm to realize interactive functions such as grasping and manipulating objects in three-dimensional space, stereo vision imaging capability is also required. Considering both requirements, the robot platform uses the same 3D stereo vision camera as the navigation module. When performing image processing tasks, the color image capture function can accomplish tasks such as face recognition, color recognition, and target tracking. When performing 3D spatial observation of the robotic arm, the depth information can be used to observe and measure the relative position of the robotic arm in the robot’s body coordinate system, thereby completing functions such as guiding the robotic arm to grasp objects.
Object recognition system: The robot uses the YOLO (You Only Look Once) method to detect objects. In the task of simulating warehouse item recognition, the robot uses a 3D stereo vision camera for shelf detection, table detection, and target detection. Before actual recognition, the robot photographs the predefined objects, then labels them by adding annotation tags and bounding boxes to each image. The robot needs to capture images from different angles, under different lighting conditions, and against different backgrounds to ensure that the model actually used matches the complex real working environment.
Human pose detection: In the human pose detection system, this project uses Carnegie Mellon University’s OpenPose as the skeleton detection method. It is a real-time multi-person keypoint detection library for body, face, hand, and foot estimation. OpenPose takes an RGB image and returns the number of people and their skeleton positions. To obtain the pointing direction of a person, the 3D coordinates of the wrist and elbow joints are needed. The OpenPose results are combined with the Point Cloud Library (PCL) to obtain the positions described in the coordinate system of the robot’s top 3D stereo vision camera, which are then converted to the map coordinate system using the TF transformation matrix. In addition, the space vector method is used to calculate which point on the ground the person is pointing at.
Robotic Arm Module
Robotic Arm Module：The robotic arm is an important means for the robot to interact with objects in space. When selecting the robotic arm, factors such as the carrying capacity of the motion module, grasping weight, and spatial operating range were fully considered, resulting in the installation of a 5-degree-of-freedom robotic arm driven by high-precision encoder servos. The arm has a total weight of about 450 g, an angular accuracy of 0.29°, and a static torque of 1.5 N·m. It is installed on the robot platform at a height of about 45 cm above the ground, and its 3D spatial range of motion is about a 40 cm radius. To better grasp objects such as cups and beverage bottles, a special palm shape was produced using 3D printing technology.
This project uses MoveIt! as the robotic arm software framework, combining robotic arm control with target detection by color detection and target recognition by image processing to achieve operation on targets. Once a target is recognized, the target is localized through a 3D point cloud to obtain its position, and inverse kinematics is solved to guide the arm to move toward the target and complete the grasp. In addition, by using MoveIt!, the robotic arm’s motion can be planned to avoid obstacles and prevent collisions with surrounding objects.