Credit: KTSDESIGN/GETTY

High performance autonomous cranes

Cranes are vital for national economic development. As under-actuated mechanical systems with fewer control inputs than degrees of freedom to be controlled, cranes cannot be operated efficiently in manual mode. Breakthroughs at Nankai have enabled more efficient robotic control of under-actuated systems such as cranes.

A crane must safely transport a lifted load to the correct position quickly and with minimal oscillation after the trolley stops. A robotic system with efficient autonomous control would greatly improve performance and safety, and has been the goal of a team led by Fang Yongchun, the dean of Nankai’s College of Artificial Intelligence.

The lack of an effective mathematical model capable of describing the transient behaviour of under-actuated systems has been a significant theoretical hurdle. Fang’s team applied energy analysis to the dynamic equation and optimized the system’s transient performance by controlling the energy conversion between the driven and non-driven states. Using this framework, they designed effective methods to address the challenges in crane control.

Another of Fang’s team’s inventions was to apply creative robotics trajectory planning to crane control. They separated the control problem of under-actuated systems into two challenges, motion planning and tracking control — improving clarity for design and analysis. This yielded new approaches to trajectory planning, tracking control, dynamic target extraction, collision prediction, and emergency braking.

Their method led to a 32-tonne automated bridge crane system that has improved transportation efficiency by more than 77%, and reduced the accident rate by 50%. It also helps significantly improve crane operators’ work efficiency, which could potentially alleviate China’s shortage of skilled crane personnel.

There is a high demand for such high precision autonomous cranes for transporting and processing nuclear waste, and for operating in hazardous environments.

The work has led to more than 30 publications in quality journals, including Automatica and IEEE Transactions, and numerous awards, including the Wu Wenjun Artificial Intelligence Natural Science Award (First Class). “We are now promoting industrialization of our robotic bridge crane, looking to broaden its use,” said Fang.

From brain-control to controlling the brain

The intersection of brain science and artificial intelligence is to play a crucial role in affecting the paradigm of thinking and lifestyle, providing new approaches to understanding and changing the world. Duan Feng, a professor of Nankai’s College of Artificial Intelligence is a stalwart in the investigation into the junction of artificial intelligence and brain science. His research team has succeeded in controlling both electronic devices and animal brains using human electroencephalographic (EEG) signals, in a sense, developing both brain-control and control-brain technologies.

Duan’s brain-control technique was demonstrated in China’s first brain-controlled vehicle, shown at Nankai University in 2014; the first time a human brain and a vehicle’s central control system were connected. The team built a complete system and achieved mind-driven vehicles capable of steering, acceleration and deceleration, and parking, all controlled by human EEG signals. The team combined the image control system with the EEG system for precision control and driver fatigue allowance. When the system detects a curve ahead, the driver’s EEG signal is identified and used to control the steering of the vehicle. When driver fatigue is detected in a specific brain region, the system performs an automated parking procedure to avoid accidents. The system has been installed on some models of sports utility vehicles produced by the Great Wall Motor Company Limited and has passed the requisite tests for going to market, with commercialization planned soon.

For the control-brain technology, Duan’s team has overcome many technical obstacles to build a rat robot control system to control a rat brain via a human brain. The rat robot’s hardware consists of an electrode, a host computer and a Wi-Fi module for receiving commands from the human brain. An electrode implanted in the rat brain electrically stimulates specific regions to remotely control the rat robot through human EEG signals. The software sets parameters for electrical stimulation through a graphic interface, and displays the results.

Using EEG signals, the team has so far controlled the rat robot to make left and right turns, as well as forward movement. In the future, the technique could be used in situations such as disaster relief and investigation in extreme environments.

With researchers across the world launching brain science and artificial intelligence initiatives, brain-inspired chips will undoubtedly become the next strategic battleground. Duan’s team is actively engaged in this research area, collaborating with RIKEN in Japan, and China’s National Supercomputer Center in Tianjin. They expect to achieve whole-brain simulation through high-performance computing, with breakthroughs in analysing and simulating brains, as well as applying brain science study results. For now, Duan’s team is developing brain-inspired energy-efficient neuromorphic chips, which will open new prospects for the chip industry.

Animal cloning through robotics

Breakthroughs in AI by Nankai researchers are also boosting life science research. In 2017, the world’s first 13 robot-cloned piglets were produced at Nankai University, demonstrating that animal clones can be created through robotic operations.

Cell micro-manipulation is a fundamental bioengineering technique that can be used to produce cells with identical genes for cloning. However, it is technically challenging, and labour and time-intensive. While there are automated systems available, they often ignore the effect on subsequent biological processes, such as cell cultivation, leading to biological results no better than those from manual operations.

New AI technologies are needed to facilitate life science research, says Zhao Xin, a professor from Nankai’s AI college, who led the robot-cloned piglet project. Zhao’s team has recently devised a new system, called an in-situ microanalysis and micro-manipulation apparatus, which integrates detection, analysis and operation. It is site-specific and minimally invasive.

Zhao introduced a mechanical model and siphon effect to the team’s AI design. In this, only minimum force is needed for high-precision automated cell orientation and cell nuclei removal. Experiments showed a significant reduction of cell damage after the process. Most importantly, the success rate of the subsequent development of cloned embryos was 21% for the robotic process versus 10% for manual operation. Furthermore, for the first time, Zhao’s team established the relationship between cell stress and subsequent cell development in cloning, and now they use success rate as a criteria for system evaluation.

By using a robotic process, Zhao’s team has completed thousands of nucleus transfer operations. In 2017, they transplanted 510 reconstructed embryos to six sows, two of whom gave birth. As of April, 2018, nine robot-cloned pigs have produced 101 progenies.

Zhao’s system has made cloning much easier. Not only does it overcome technical barriers to biological cloning, but it also frees researchers from manual micro-manipulations. The in-situ microanalysis and micro-manipulation apparatus has been applied to other animals, such as sheep, zebrafish, and mice. These studies pave the way for other biological robotic platforms and open a new path for improving biological development after cell manipulation.

This multi-disciplinary study by Nankai scientists straddles the interface of AI and life science research. Their work was selected among the 2018 China Ten Great Technology Progress in Intelligent Manufacturing, and gained wide media coverage.

Bioinformatics and genomic

Big data analysis

Another way that AI can assist life science research is demonstrated by the cutting-edge work on statistical machine learning and bioinformatics by Zhang Han, who heads the Data Science and Computer Intelligence Laboratory at Nankai’s AI college.

Through machine learning, Zhang’s team has made in-depth studies of bacterial genome arrangement rules and the effects of operons — a DNA sequence containing a group of genes controlled by a single promoter — and gene expression on metabolic pathways.

They learned the arrangement rules by analyzing the transcriptional factors in operons and regulons, a group of co-regulated operons. It is known that bacterial genes of the same pathway tend to group into operons, but little is known about what determines the global arrangement of those genes in a genome beyond the operon level.

Zhang’s team found that genomic locations of bacterial operons are constrained by biological pathways encoded in the genome, and are also influenced by the activation frequencies of pathways. Generally, operons of the same pathways tend to cluster together. These fundamental findings were published in the Proceedings of National Academy of Sciences.

The team has applied big-data analysis methods to the gene expression of neurological diseases and has designed a technique to identify disease-correlated genes. They investigated deep learning training methods for identifying features in genetic data indicating disease, and mined that data, based on a weakly-supervised learning framework.

To address the dynamic evolution of gene expression, an integrated analysis method was designed. These data analysis methods can be applied to bioinformatics issues related to gene expression in neurological diseases, protein sequence analysis, and classification of carbohydrate active enzymes.

Neuromuscular modelling for stroke diagnosis and rehabilitation

Electro-physiological signals from the brain and muscles, known respectively as EEG and EMG (electromyography) signals, provide important information about the neuromuscular system and can be used to diagnose neurological defects, and control prostheses and rehabilitation robots.

Modelling the dynamic relationship between the multiple brain regions and muscles is essential for understanding brain-muscle interaction and for quantitative prediction of limb and joint movements. Such models can be invaluable in the diagnosis and rehabilitation of stroke patients. In Nankai’s College of Artificial Intelligence, a research team led by Han Jianda has made significant advances in the development of models based on EEG and EMG signals.

Han’s team has developed an EMG-driven state-space model for estimating the velocities and angles of human joints based on surface EMG signals alone. The kinetic variables of motion are expressed as a function of neural activation levels and the estimated results can be used to control a robotic manipulator, making it perform tasks at the will of the operator.

The team has also developed a non-invasive EEG-based brain-computer interface that recognizes visually expressed movement intentions and can be used to trigger functional electrical stimulation for rehabilitation of stroke patients. The research results will be tested in small-scale clinical trials.

To address the dynamic evolution of gene expression, an integrated analysis method was designed. These data analysis methods can be applied to bioinformatics issues related to gene expression in neurological diseases, protein sequence analysis, and classification of carbohydrate active enzymes.

Neuromuscular modelling for stroke diagnosis and rehabilitation

Electro-physiological signals from the brain and muscles, known respectively as EEG and EMG (electromyography) signals, provide important information about the neuromuscular system and can be used to diagnose neurological defects, and control prostheses and rehabilitation robots.

Modelling the dynamic relationship between the multiple brain regions and muscles is essential for understanding brain-muscle interaction and for quantitative prediction of limb and joint movements. Such models can be invaluable in the diagnosis and rehabilitation of stroke patients. In Nankai’s College of Artificial Intelligence, a research team led by Han Jianda has made significant advances in the development of models based on EEG and EMG signals.

Han’s team has developed an EMG-driven state-space model for estimating the velocities and angles of human joints based on surface EMG signals alone. The kinetic variables of motion are expressed as a function of neural activation levels and the estimated results can be used to control a robotic manipulator, making it perform tasks at the will of the operator.

The team has also developed a non-invasive EEG-based brain-computer interface that recognizes visually expressed movement intentions and can be used to trigger functional electrical stimulation for rehabilitation of stroke patients. The research results will be tested in small-scale clinical trials.