1. Voice Module
The voice module consists of two parts — sound collection and sound output。For natural language recognition, the robot uses the CMU Pocketsphinx speech recognition model based on the Sphinxbase library。The speech recognition system uses the latest version, “sphinxbase-5prealpha”, to build the application。GStreamer is used to automatically split the input audio into the speech to be recognized, and provides services to start and stop recognition。The speech recognition system requires specific language model and dictionary files, which can be automatically built from sentence corpora of natural language。For text-to-speech (TTS), the robot uses the CMU Festival system and the ROS Sound_Play package。
To improve speech recognition accuracy in noisy environments, the Sphinxtrain tool is used within the CMU Pocketsphinx speech recognition model to train sentence recordings in noisy environments。The Sphinxtrain tool uses a large number of the above recordings as a database to extract the sounds of noisy environments。The obtained parameters are then used to replace the original parameters for better speech recognition detection。
2. Vision Module
Vision image processing needs to realize real-time classification and recognition of targets, which requires the image sensor to have good dynamic performance and high image quality。In addition, for some functions such as ranging or spatial stereo vision processing, depth information is also required。Considering both, the Io platform is equipped with a 3D stereo vision camera。
Face information recognition: The OpenCV library is used to implement face detection, training, and recognition。Models from the OpenVINO model library can also recognize facial expressions, age, gender, and orientation。
Object recognition system: The robot uses the YOLO method to detect objects。In the task of simulating warehouse item recognition, the robot uses a 3D stereo vision camera for shelf detection, table detection, and target detection。Before actual recognition, the robot photographs the predefined objects, then labels them by adding annotation tags and bounding boxes to each image。The robot needs to capture images from different angles, under different lighting conditions, and against different backgrounds to ensure that the model actually used matches the complex real working environment。
Human pose detection: In the human pose detection system, this project uses Carnegie Mellon University’s OpenPose as the skeleton detection method。It is a real-time multi-person keypoint detection library for body, face, hand, and foot estimation。OpenPose takes an RGB image and returns the number of people and their skeleton positions。To obtain the pointing direction of a person, the 3D coordinates of the wrist and elbow joints are needed。The OpenPose results are combined with the Point Cloud Library (PCL) to obtain the positions described in the coordinate system of the robot’s top 3D stereo vision camera, which are then converted to the map coordinate system using the TF transformation matrix。In addition, the space vector method is used to calculate which point on the ground the person is pointing at。
3. Storage Module
The storage module uses high-speed solid-state storage media with read/write speeds up to 400 MB/s. This ensures that Io is plug-and-play while the system runs smoothly and improves development efficiency。The storage module is installed with the Ubuntu system, ROS, and related application software, and is configured with complete drivers and functional packages, including all functions for image processing, speech recognition, robotic arm motion simulation, indoor mapping, and navigation simulation。It supports both UEFI and Legacy dual-boot modes, can quickly boot the system via a USB 3.0 connection, and supports one-click system restore。
The system simulation platform is configured with a 7-degree-of-freedom robotic arm motion planning case, capable of completing MoveIt! motion planning calculations。It is also configured with two sets of virtual home environments that can fully implement robot SLAM mapping and autonomous navigation。It supports gmapping, karto 2D SLAM, and rtabmap 3D map construction; supports various path planning algorithms, and the navigation parameters can be adjusted independently。
4. Communication Module
A USB 3.0 or higher port is used to directly supply power and carry out data communication; the platform comes with a 150M Wi-Fi module and supports remote control。
All of our products come with a one-year on-site warranty, covering hardware damage and data loss caused by non-human factors。
One free original-factory data rescue service is provided each year。