Recently, Associate Professor Chen Xudong from School of Physics of Nankai University, Professor Lu Tongbu from Tianjin University of Technology, and Academician Zhang Jin from Peking University, who were listed as co-corresponding authors, published the latest research results of novel retinal morphology devices for integrated sensing and computing on the internationally renowned journal Science Advances. The team realized nonvolatile linear modulation of responsivity in positive and negative regimes by replacing externally applied gate voltages with ion-induced local fields. A sensor array was fabricated to constitute a convolutional neural network (CNN) involving three kernels, thus realizing the image classification and encoding and so forth in sensor at the hardware level. The operating spectra of this retinal morphology device range from 400 to 1,800 nm. 

Machine vision has important applications in fields such as autonomous driving, human-machine interaction (HMI), and Industry 4.0, and plays an indispensable role in the era of artificial intelligence. However, its potential in applications such as high-speed target tracking and recognition is seriously limited because of the huge delay and energy consumption generated by traditional machine vision based on the von Neumann architecture in processing massive data from the sensor. A solution for the bottleneck of computing at edge and energy efficiency is to deploy neural networks directly on the sensor end for integrated sensing and computing, and develop neuromorphic machine vision. In recent years, the retinal morphology devices for integrated sensing and computing, analogous to the human visual systems, have emerged as a forefront and hotspot in the field of optoelectronic devices. How to implement MAC (multiple and aggregation) operation and local storage of weights (responsivity) in the sensor array is a key issue that must be addressed in the development of devices for integrated sensing and computing.

This research utilized aerosol printing technology to construct an ion photoelectric transistor device based on core-sheath SWNT@GDY nanotubes. This device accurately controlled the concentration of lithium ions captured in GDY and regulated the ion-induced local fields, so it could selectively capture photogenerated electrons or holes in single-walled carbon nanotube (SWNT) channels and be based on the photogating effect to realize linear modulation of responsivity of the device in positive and negative regimes. Thanks to the excellent storage performance of graphene, lithium ions can be stably stored in it leading to nonlinear modulation of responsivity. At the same time, carbon nanotubes exhibit good light absorption characteristics in the UV-visible-infrared bands, thus the device's operating spectral range covers 400-1,800 nm. Furthermore, a 3×3×3 sensor array was fabricated to constitute a convolutional neural network (CNN) involving three kernels, which could realize the image processing, classification and encoding and so forth in the sensor at the hardware level as well as demonstrate the local storage of weights (responsivity) in the neural network.

The paper summarizing the relevant research results titled Broadband Sensory Networks with Locally Stored Responsivities for Neuromorphic Machine Vision was published in Science Advances. The first author affiliation of the paper is Tianjin University of Technology, and the second author affiliation of the paper is Nankai University. Zhang Guoxin, a master student of Tianjin University of Technology, and Zhang Zhicheng, a doctoral student of Nankai University, are the co-first authors of the paper. Cooperation organizations also include Suzhou Institute of Nano-Tech and Nano-Bionics (SINANO), Chinese Academy of Sciences.

Paper link:

https://www.science.org/doi/10.1126/sciadv.adi5104

（Edited and translated by Nankai News Team.)