PyCon HK 2023-Python in Smart Logistics

很棒的经历，很棒的伙伴，很棒的组织，很棒的体验。

PyCon HK 2023

Speech Script

Good afternoon everyone, I want to thank the community and PyCon. I am delighted to have the opportunity to share with you my experience of using Python to build intelligent robot dispatching systems in my work.

My name is Kang Hao. I joined LSCM in May this year.I studied Mechatronic Engineering before 2020. During my student years, I took on the redesign project of a towing tank tugboat. As shown in the picture, the tugboat is used in ship design for ship simulation, verifying the rationality of designs. For example, this picture is the measuring bridge of the tugboat, used to fix the ship model, and the traction required for the operation of the ship model is read from the dynamometer on the bridge. As the equipment was manufactured at the beginning of the century, the built-in control and measurement systems can no longer meet modern ship model requirements.

My task was to redesign the mechanical structure and electrical system of the tugboat to make it run faster and more stably, better expand various experimental equipment, and a crucial approach was to use simulation software to evaluate the structure.

As we all know, Hong Kong has always been an important port and center for logistics in Asia. The logistics industry is one of the four pillar industries and contributes one-fifth of the GDP. Many institutions predict that the Hong Kong logistics market will continue to grow in the next five years, and many companies will compete.

We should all be familiar with logistics. In modern production and commercial fields, one important aspect is cash flow, and the other is logistics. With the development of the times, logistics methods have also undergone revolutionary changes. Whenever I see Logistics 1.0, I think of a modern poem: "In the old days, carriages were slow, and letters were far away – one life is enough to love one person". But now, it’s very fast, as shown in Logistics 4.0. We can use automated equipment, such as AGVs used in the warehousing environment, automated forklifts, and unmanned delivery vehicles outdoors. This greatly improves efficiency and reduces costs. The Hong Kong government has also proposed a series of policies aimed at Logistics 4.0 in recent years, promoting industry development.

So how can Python be applied in such a rapidly developing and changing industry? Please imagine this scenario. When you are in a modern warehouse, you need to be able to control various types of robots, such as common AGVs, automated forklifts, robotic arms, etc. At the same time, you need to communicate with various peripherals in the space, such as rolling doors, conveyors, and elevators, some of which are controlled by PLCs, and some by industrial computers. You need to be able to interface with various protocols well. Don’t forget, you need to give the on-site operators an intuitive and convenient operation interface, as well as on-site data monitoring and collection. Finally, you may also need to upload important production data to cloud servers for subsequent analysis. What’s more surprising is, what if you are the only developer?

So I took these questions to a wise man. I wanted a computer language that has these characteristics… Is there such a language? He told me, that language is Python.

In fact, there are many robot dispatching systems on the market. Perhaps due to technical limitations and personal habits at that time, many systems have issues like overly distributed functions. We all know that to complete a full set of operations, we first need to give the robot a map, edit sites on the map, trigger corresponding tasks, and the system controls the robot to execute. Many systems need the support of 2~3 different programs to achieve the above functions, and many programs can only run on industrial computers, without a friendly control panel. Or they do not pay much attention to the ease of use of the system, which increases the difficulty of operation and, to some extent, increases the chance of errors. I’m sorry that I left my previous company in the first half of the year, so today I can’t show the system interface here, but if you are interested, you can check out my speech last year, which touched on it.

Now look at another scenario. On the robot side, we need to write a layer that interacts with the system, responsible for the robot’s state machine and action parsing and feedback. I’ve seen very good C# programs to achieve this, with clear and rigorous logic. But we know that the ROS system cannot directly interact with C# programs. It must go through rosbridge to convert ros topics into websockets. Of course, ros_bridge is also written in Python.

So why not directly use Python to write this layer of logic? Communicate directly with ROS, fewer intermediaries, and a lower error rate.

Therefore, based on the requirements and the Python ecosystem, we designed the robot dispatching system as shown below. Operators can easily operate robots and systems through tablets or PCs, and all operations are on one website, including using Bluetooth to control robots to build SLAM maps or directly import QRcode maps, bind maps with different robots, edit maps, create tasks, control tasks. At the same time, the system supports multiple accounts with different operational permissions, for example, operational permissions only support task control, more detailed permission division. The dispatching system complies with the VDA5050 protocol specification and communicates with the robot state machine through MQTT. Different data uses are stored in MySQL and InfluxDB. MySQL stores task content, including sites, actions, repetition modes, etc. InfluxDB is used to store necessary time-series data because its primary key is a timestamp, which is convenient for subsequent diagnosis of robot failures and life expectancy prediction. Some designs can be seen in the blog. At the same time, the system shares data through MQ communication for different public modules to implement common path planning, robot control, exception handling, etc., such as task breakpoint recovery, balanced distribution of robots in different areas, etc. Some algorithms can be easily found in Scikit-learn, which I will go into detail later.

Simultaneously, using the concept of OOP, we designed robot simulation models that implement multiple navigation methods such as SLAM, QR code, and magnetic navigation. It also realizes multiple action executions. The simulation model can be abstracted into various robots. For example, this is an AMR model, which includes several parts. It can easily simulate and evaluate on-site solutions without buying many AMR entities because each has an average price of about 100,000 Hong Kong dollars. It can also be used for academic research. There are many MAPF simulation platforms now, based on C# or Java, but the running platform is limited. But this can solve the problem well.

One of the reasons why everyone loves Python is because of its ease of use, and the most concern is probably Python’s performance. There is an old saying in China called "make the best use of everything", how is this idea reflected in the project? Starting from the robot side, the state machine is a typical I/O intensive program. It needs to receive various sensor data and execution results for judgment, and send information out through different topics. Python has a complete ecosystem and a friendly implementation method in terms of asynchrony. For example, when communicating with ROS, use asyncio and rclpy in cooperation, wrap the callback functions of different topics into asynchronous, and use aiomqtt and websockets to asynchronously send messages to avoid blocking. All you need to do is read the documentation for implementation.

Secondly, make good use of built-in data structures, because many have been optimized. A typical example is to use a queue to store actions and a stack to trace available states. Optimized dictionaries can quickly index the callback functions corresponding to topics, interfaces corresponding to nodes, and various nested data structures. This meets performance requirements.

The last one is to use Nuitka to convert Python programs into bytecode. It’s surprising that Nuitka also supports rclpy. It only takes one command to package the source code, avoid tampering, improve the running speed of Python programs in an ARM environment, and support cross-platform.

For the dispatching system, the first is to use algorithms as much as possible to reduce time complexity. Here are three examples. First, use a priority queue to store tasks to be executed and AGVs that meet the state. Sorting often happens when related information is updated. For example, when receiving a task or AGV information, select the most suitable task and AGV through the selector, such as the AGV closest to a certain task starting point, the one with the most power, or the one with a longer idle time to execute the task.

Use KDTree for fast convergence to find the nearest site for AGV to occupy or task site to find the nearest AGV to execute. At the same time, robots of different sizes also occupy different spaces. To avoid collisions, you need to occupy their surrounding sites. This condition would take a lot of time and resources to use traversal, but using KDTree, a binary search algorithm in space, can be implemented quickly. We can implement this algorithm ourselves, or directly call the relevant algorithm in scikit-learn, thanks to Python!

Also, separate data of different heat levels for processing. Those that update quickly and are very important, such as robot status, position, task feedback, are stored in memory and shared in Python, then published externally and written into Redis, etc., at a slower speed, focusing the main effort on the most valuable problems.

In the server, use PyPy to run the program, which supports the original ecosystem well and optimizes concurrent execution. Even without a distributed structure, it can easily handle the control of hundreds or even thousands of robots.

Due to time constraints, I can only briefly introduce how machine learning is applied in the system now. More detailed content may be shared with you next year. Machine learning is mainly used in these four aspects in smart logistics. Currently, I mainly use the DBSCAN clustering algorithm in unsupervised learning to dynamically evaluate path costs for conflict avoidance and conflict resolution. For instance, in congested areas, a higher path cost will deter more AGVs from entering. Among the AGVs in the congestion area, priority is given to relieve the ones with the highest rank, thereby achieving congestion control.

Lastly, I want to emphasize that Python is not just a tool for us, it is more like a partner. It helps us to think in a more structured way, to build things in a more modular way, and to solve problems in a more Pythonic way. Let’s embrace Python and let it help us to make our work and our world a better place.

Thank you for your attention. If you have any questions or comments, I would be happy to discuss them with you.