Oscillation neuron based on a low-variability threshold switching device for high-performance neuromorphic computing

Yujia Li , Jianshi Tang, , Bin Gao, Xinyi Li, Yue Xi, Wanrong Zhang He Qian, and Huaqiang Wu,

1Faculty of Information Technology, Beijing University of Technology, Beijing 100124, China

2Institute of Microelectronics, Beijing National Research Center for Information Science and Technology (BNRist), Tsinghua University,Beijing 100084, China

3Beijing Innovation Center for Future Chips (ICFC), Tsinghua University, Beijing 100084, China

Abstract: Low-power and low-variability artificial neuronal devices are highly desired for high-performance neuromorphic computing.In this paper, an oscillation neuron based on a low-variability Ag nanodots (NDs) threshold switching (TS) device with low operation voltage, large on/off ratio and high uniformity is presented.Measurement results indicate that this neuron demonstrates self-oscillation behavior under applied voltages as low as 1 V.The oscillation frequency increases with the applied voltage pulse amplitude and decreases with the load resistance.It can then be used to evaluate the resistive random-access memory (RRAM) synaptic weights accurately when the oscillation neuron is connected to the output of the RRAM crossbar array for neuromorphic computing.Meanwhile, simulation results show that a large RRAM crossbar array (> 128 × 128) can be supported by our oscillation neuron owing to the high on/off ratio (> 108) of Ag NDs TS device.Moreover, the high uniformity of the Ag NDs TS device helps improve the distribution of the output frequency and suppress the degradation of neural network recognition accuracy (< 1%).Therefore, the developed oscillation neuron based on the Ag NDs TS device shows great potential for future neuromorphic computing applications.

Key words: threshold switching; Ag nanodots; oscillation neuron; neuromorphic computing

1.Introduction

A resistive random-access memory (RRAM)-based neural network has been extensively studied as a promising solution to overcome the von Neumann bottleneck faced in conventional artificial intelligence (AI) hardware[1−4].As inspired by a biological neural network, an artificial neural network consists of synaptic and neuronal devices.In order to improve speed and power efficiency, the RRAM crossbar array, which can significantly accelerate the vector-matrix multiplication,has been developed to implement artificial synapses[5−8].On the other hand, a neuronal device is needed at the end of each crossbar bit line (BL) to convert the weighted sum current of the analog RRAM synapses into spikes to transmit information to the next layer of neurons.Here integrateand-fire neurons built with CMOS circuits are typically employed[9].However, such complex CMOS neurons would occupy a much larger footprint than the BL pitch of the crossbar array, which causes serious column pitch matching problem[10, 11].

Recently, a compact oscillation neuron based on a metal–insulator transition (MIT) threshold switching (TS) device was proposed as a more scalable artificial neuron[12−16].Compared to the CMOS neuron, an oscillation neuron has the benefits of small size and simple circuit structure, which is appealing for large-scale neuromorphic system applications.However, the on/off ratio of a typical MIT TS device is small(~102), which cannot be used for a large RRAM crossbar array(e.g., 12 × 1 array)[15].Moreover, the high operation voltage of the MIT TS device may disturb the weights of the RRAM synapses and also increase the power consumption[16].Alternatively, TS devices based on electrochemical metallization(ECM) filaments have attracted considerable attention due to their simple structure, large on/off ratio, and low operation voltage[17−23].However, the uniformity of typical ECM TS devices is relatively poor, which may affect the accuracy of artificial neural networks.More recently, we have developed a high-uniformity HfO2-based TS device with patterned Ag nan-odots (NDs) as the high-performance selector, which shows low leakage current (< 1 pA), high on/off ratio (> 108), and high endurance (> 108cycles)[24].

Fig.1.(Color online) (a) Schematic diagram of a typical artificial neural network.(b) Circuit implementation of the oscillation neuron with a TS device.

Fig.2.(Color online) (a) TEM image of the Ag NDs TS device.(b) Schematic illustration of the threshold switching process in the device.(c) Typical current–voltage (I–V) curves for the Ag NDs TS device.(d) Cumulative probability of Vth and Vhold distributions for the Ag NDs TS device.(e) Endurance test of the Ag NDs TS device with over 108 cycles.(f) Measured oscillation waveform of the oscillation neuron.

In this work, we further implement an oscillation neuron using the HfO2/Ag NDs TS device.This neuron exhibits selfoscillation behavior at low applied voltage (1 V), where the oscillation frequency increases with the applied voltage and decreases with the load resistance.In addition, it can work with a large RRAM crossbar array (> 128 × 128) owing to the high on/off ratio (> 108) of Ag NDs TS device.Moreover, in the neural network simulation, a high recognition accuracy (loss <1%) can be achieved by using this oscillation neuron because of its high uniformity.

2.Results and discussion

Fig.1(a) illustrates the schematic diagram of a simple artificial neural network.When the input voltages are applied to the crossbar synaptic array, the weighted sum currents are integrated by the neurons at the end of each column (BL) and trigger output spike firing when they reach the neuron thresholds.The circuit implementation of the oscillation neuron based on the TS device is shown in Fig.1(b).The load resistance (RL) represents the RRAM synaptic weight connected in series with the TS device.Also,CLis the load capacitance including parallel capacitance and parasitic capacitance at the neuron node.Initially, the TS device is in the off state (Roff).When applying an input voltage pulse (Vin), the voltage mainly drops on the TS device sinceRoff>RL, andCLstarts to charge.WhenVinis larger than the threshold voltage of the TS deviceVth, it turns to the on state (Ron), and thenCLstarts to discharge since the voltage drop on the TS device is reduced (Ron>RL>>Ron, the ideal oscillation frequencyfcan be described as[13]:

Eq.(1) shows a one-to-one correspondence betweenfandRLat a certainCLandVin.Therefore, the oscillation frequency can be used to represent the weight of the RRAM synapse accurately.

In this study, we demonstrate the oscillation neuron using the Ag NDs TS device based on ECM filaments.The devices were fabricated with a cell size of 10 × 10μm2.The bottom electrode was patterned by photolithography and deposited by sputtering of 5 nm Ti and 50 nm Pt.An 8 nm thick HfO2dielectric was deposited by atomic layer deposition(ALD) at 250 °C.The Ag NDs with diameters of 50 nm were patterned by e-beam lithography (EBL) and deposited by sputtering.A 40 nm-thick Pt was deposited as the top electrode.The transmission electron microscope (TEM) image of the Ag NDs TS device is shown in Fig.2(a).Fig.2(b) exhibits the schematic illustration of the threshold switching process in the device.In the initial state, there is no conductive filament in the dielectric layer, and the device is in the high-resistance state (HRS).When applying a voltage (Vappl) larger thanVth,the electric field is locally enhanced in the areas with Ag NDs,so the Ag atoms are easier to be ionized as the ion source(Ag → Ag++ e−) for diffusing toward the bottom electrode,which leads to the formation of metallic filaments and turns the device to a low-resistance state (LRS).In this device, the Ag filaments are thin and unstable due to the small amount of Ag ions.Spontaneous rupture of the filaments occurs immediately whenVapplgoes belowVhold, which turns the device back to HRS.In this process, the Ag atoms form clusters on the trace of filaments.Owing to the highly ordered Ag NDs,Ag filaments tend to form at the same positions and the formed filaments would have similar morphology, in different operation cycles or different devices.Fig.2(c) shows the typical current–voltage (I–V) curve of the Ag NDs TS device under voltage sweeps between 0 and 1 V.This device exhibits alow leakage current less than 1 pA and large selectivity over 108.TheVthandVholdof Ag NDs TS device are 0.6 and 0.2 V, respectively, which are carefully tuned to work with the RRAM synapse.In order to evaluate the uniformity of the Ag NDs TS device, the distributions ofVthandVholdare analyzed, and the cumulative probability ofVthandVholdis shown in Fig.2(d).The coefficient of variation is defined asCV=σ/μto evaluate the variation, whereμandσare the mean and standard deviation, respectively.This Ag NDs TS device exhibits excellent uniformity (CV< 10%) compared to other TS devices based on ECM filaments[24].The endurance test of the Ag NDs TS device is shown in Fig.2(e).In this measurement, the device is repeatedly turned on with SET pulses ofVset= 1 V andtset= 10μs, and then relaxed to the off state, which is read with a small pulse ofVread= 0.1 V andtread= 10μs.The onstate current (Ion) is limited to 10μA.It is found that the Ag NDs TS device exhibits a high endurance of over 108cycles.To implement an oscillation neuron, we connect the Ag NDs TS device in parallel with a load capacitance (CL) and then in series with a load resistor (RL) following the circuit configuration in Fig.1(b).Fig.2(f) shows the measured oscillation waveform of the neuron whenRL= 50 kΩ,CL= 750 pF andVin=1 V.The test result shows that this oscillation neuron can output a continuous oscillation signal, whenVin,RLandCLare fixed.In addition, the Ag NDs TS oscillation neuron shows a certain time delay before its stable oscillation, which leads to a higherVoutin the first peak.It is owing to the turn-on delay time of the TS device[24].The higher first peak may have an adverse effect on the synaptic weight precision by distorting the oscillation waveform, which can be minimized by further improving the TS device switching speed.

Fig.3.(Color online) (a) Oscillation waveforms of the oscillation neuron with different Vin when RL = 50 kΩ, CL = 750 pF.(b) The oscillation frequency as a function of Vin. (c) Oscillation waveforms of the oscillation neuron with different RL when Vin = 1.2 V, CL = 750 pF.(d) The oscillation frequency as a function of RL.

Fig.4.(Color online) The oscillation frequency as a function of the RRAM crossbar array size under different on/off ratios of the TS device.

More systematic studies on the oscillation neuron characteristics are performed as shown in Fig.3.Fig.3(a) shows the oscillation waveforms at different input pulse voltages whenRL= 50 kΩ andCL= 750 pF.The oscillation frequencies are 31.6, 40.2, 46.9 and 51.4 kHz forVin= 1, 1.2, 1.4, and 1.6 V, re-spectively.It is found that the oscillation frequency increases as a function of the pulse amplitude ofVinas described in Eq.(1), and shows good consistency with the simulation results,as shown in Fig.3(b).Similarly, Fig.3(c) shows the oscillation waveforms of the Ag NDs TS devices with differentRLvalues(i.e., different synaptic weights) whenVin= 1.2 V andCL=750 pF.The oscillation frequencies are 40.2, 30.5, 26.1 and 16.8 kHz forRL= 50, 80, 100 and 150 kΩ, respectively.The oscillation frequency decreases asRLincreases, as shown in Fig.3(d).The test results indicate that this Ag NDs TS oscillation neuron exhibits self-oscillation behavior, and the output oscillation frequency can be used to sense the weight of the RRAM synapse accurately when the neuron is connected to the crossbar array.

Fig.6.(Color online) (a) The structure of MLP neural network.(b) Simulation results of the MNIST recognition accuracy loss as a function of the variability of the TS device.

Based on experimentally derived device data, a model of the RRAM crossbar array is developed to evaluate synaptic weights of different array sizes.Then SIPCE simulation is used to investigate the relationship between the oscillation frequency and the RRAM crossbar array size.In order to simplify the simulation process, the resistance RRAM device is fixed at an intermediate resistance state (RL= 500 kΩ), andVinis applied to all the word lines (WLs) in parallel.The results are shown in Fig.4.If the on/off ratio of the TS device is less than 100, only a limited range of the weighted sum could meet the criterion for oscillation (Roff>RL>Ron).It means that the oscillation neuron can only be used for small crossbar arrays(< 16 × 16).With the increase of on/off ratio, the oscillation neuron can work with a wider range of load resistance.Therefore, the weighted sum in a larger RRAM crossbar array can be successfully identified by distinguishable oscillation frequency.As in the case of our present Ag NDs TS device with on/off ratio > 108, it can work with a large RRAM crossbar array of size > 128 × 128.

In order to investigate the impact of TS device variation on the oscillation neuron, the oscillation frequency distribution with differentCVunder the same experimental conditions is shown in Fig.5(a).The uniformity of the TS device deteriorates with the increase ofCV, which leads to a large variation of the oscillation frequency under the same load condition.Fig.5(b) shows the oscillation frequency distribution of differentRLwith different TS devices.Compared with the Ag thin film device (CV~ 30%)[23], the output frequency distribution of the Ag NDs TS neuron (CV~ 7%) is more concentrated.Therefore, the oscillation neuron based on the Ag NDs TS device can achieve higher accuracy for neuromorphic computing.

In order to further analyze the impact of the TS device’s uniformity on the artificial neural network, a multi-layer perceptron (MLP) of 784 × 200 × 10 is simulated to classify the handwritten digits in the Modified National Institute of Standards and Technology (MNIST) dataset, as shown in Fig.6(a)[25].The simulation results displayed in Fig.6(b) indicate that the increase inCVleads to a dramatic degradation of the recognition accuracy, especially whenCV> 15%.There-fore, the oscillation neuron based on the high uniformity Ag NDs TS device developed in this work is beneficial to reduce the network accuracy loss.

3.Conclusion

In conclusion, we have demonstrated a reliable oscillation neuron using the low-variability Ag NDs TS device for high-performance neuromorphic computing.This neuron exhibits self-oscillation behavior at low applied voltages down to 1 V.A systematic study on the oscillation characteristics reveals that the oscillation frequency increases with the applied voltage and also the synaptic conductance connected as the load resistor.The high uniformity and large on/off ratio of the Ag NDs TS device enable the oscillation neuron to reduce the neural network accuracy loss (< 1%) and make it applicable to a large-scale RRAM crossbar array (> 128 × 128).The developed oscillation neuron hence has great potential for future neuromorphic system applications.

Acknowledgements

This work was supported in part by China Key Research and Development Program (2016YFA0201800), and the National Natural Science Foundation of China (91964104,61974081).