Bit slicing approaches for variability aware ReRAM CIM macros

3.1 Peripheral circuits

Figure 2 shows the selection circuits (a) as well as the ADC (b). The select circuitry for the BL and SL are structurally the same, but connected to different control signals and peripheral blocks for BL (marked in orange) and SL (marked in blue). The left branch uses a transmission gate to connect the RESET/SET drivers to the BL/SL respectively. Those drivers are required to program the VCM cells. At the opposite side of the active driver, the BL/SL is pulled to ground via an low ohmic NMOS transistor. The right branch of the select circuit is connected to the read driver/sensing stage for the BL/SL side. The inset in gray shows the internal structure of the used single stage differential amplifier (DA). To increase the precision in the read path we have removed the transmission gates to reduce the IR drop resulting from the series resistance of the transmission gate. Instead, power gating is used to set this path high ohmic. Power gating is achieved via a PMOS transistor between the supply voltage and the supply voltage pin of the sensing stage (right side of Figure 2(a)). By removing the supply voltage the buffers in the read drivers are set high ohmic. Also, the ADC is set high ohmic if the supply voltage is reduced (compare Figure 2(b)). The ADC circuit is inspired from [33, 36], as the sensing stage from [33] is combined with the voltage controlled oscillator from [36].

Figure 2:

Circuit blocks of the Bitline and Sourceline select circuits (a) and the ADC stage (b). In (a) signal names without curly braces are used in the BL and SL select circuits. For the signals in curly brackets the orange/blue colored signals are applied in the BL/SL select circuit. The gray inset shows the internal structure of the differential amplifier (DA).

A VMM is performed, by first turning on the read drivers. This is done by setting the ‘MAC Control’ signal and the ‘READ’ signal to ‘1’. The read drivers are implemented as digital buffers and apply a voltage of 2.2 V via the SL select circuit to the 1T1R crossbar. The ADC keeps the voltage at the output of the crossbar at 2 V, resulting in a voltage drop of 0.2 V across the 1T1R elements. Then, the logical input vector is read in and applied via the Wordlines (WL₁ … WL_N). The Wordline drivers (or input DACs) are realized as digital buffers, either applying 0 V or V_DD (5 V) to the transistor gates. This reduces the required driving power for the WL buffers, as they only have to drive a certain number of transistor gates, depending on the number of selected columns. The input vector is applied serially and fed through a shift register, to be applied in parallel to the WL drivers (see Figure 1). The correct column is chosen via the ‘SELECT’ signal, which is generated via a demultiplexer, filled by another shift register. In this way, single columns, multiple columns or all columns can be selected for an evaluation cycle. The maximum number of columns, which can be activated at the same time, depends on several factors, e.g. the layout size of the ADC, or the requirements of the application. For the highest throughput, all columns should be active at the same time, requiring one ADC per column. Our chosen approach allows for complete flexibility as to the number of activated columns. If multiple columns share a single ADC, e.g. every four columns have one ADC, the ‘MAC Select’ signal can be used to switch between these four columns. This approach is useful, if the columns should be weighed differently in the digital periphery as in the first cycle all first bit positions (2°) can be evaluated, in the second cycle all second bit positions (2¹) can be evaluated and so on. Via the ‘MAC Select’ signal one can also switch between different numbers of columns to be evaluated. After all components have been activated, they are kept active for the evaluation time, which can also be chosen in a flexible fashion depending on the required accuracy. Programming of cells is performed individually, via the left side path of Figure 2(a). To perform a write operation, first the ‘WRITE’ signal is set to one and column is chosen via the ‘SELECT’ signal. As for the VMM, the correct row can be chosen via the logical input vector by only applying a ‘1’ to the programmed cell and applying ‘0’ to all other rows. Depending on the switching direction, the SET/RESET driver is connected at the BL/SL side while the SL/BL is set to ground via an NMOS transistor. The configuration of the periphery for the different operation modes is summarized in Table 1. The signals in bold print are global signals, that are only required once for all columns.

The full ADC, shown in Figure 2(b), can be split into three parts, namely the voltage stabilization stage (marked in blue in Figure 2), the ring oscillator stage and the counter stage. The sensing stage clamps the BL voltage to the reference voltage V_Ref = 2 V during the VMM operation to achieve a constant voltage drop across the 1T1R crossbar, which increases the resolution of the ADC [33]. With the read driver applying 2.2 V and V_Ref set to 2 V, 0.2 V drop over the 1T1R cells. This is the same voltage configuration for which the conductance levels are verified, to prevent the IV nonlinearity of the VCM cells from affecting the results. Via the differential amplifier it is ensured, that the resulting current levels are spaced linearly. N₁ mirrors the current from N₀ into the voltage controlled oscillator (VCO) stage, where it is converted into the supply voltage of the VCO, V_{DD, VCO} by R_Supp. Thus, the VCO oscillates at different frequencies, depending on its supply voltage. To improve the counting of the pulses, the output signal of the VCO is buffered through an inverter chain. As the ADC converts an amplitude encoded signal (crossbar output current) into a time encoded signal (pulse train to be counted), higher accuracies can be achieved by increasing the conversion time [36]. The time for which the pulses are counted depends on the required precision as will be shown in the next section.

3.2 Basic performance characteristics

In order to study the effects of the different approaches to bit slicing, first a baseline case needs to be defined. Here, we choose a simulation in which 16 rows with binary devices are read out with a single 4-Bit ADC. The 16 rows are chosen to not put to high requirements on the devices and the ADC. For the chosen array transistors (W = 10 μm, L = 500 nm) and the device states N_{disc, LRS/HRS} = 0.904·10²⁶·1/m³/0.148·10²³·1/m³ the resulting resistances of the 1T1R structure are 4.65 kΩ/95.97 kΩ or 214.95 μS/10.42 μS. In the JART VCM v1b model the switching filament is simplified via a well conducting oxygen reservoir, the plug region, and the variable resistance disc region. The state variable of the model (N_disc) can then be used to set the resistance of the devices. In the 1-Bit case HRS represents the logical ‘0’ and the LRS represents the logical ‘1’. For 2-Bit the HRS still represents level ‘0’, while the LRS represents level ‘3’. Both values were evaluated at a read voltage of 0.2 V in the RESET direction and a gate voltage of 5 V. For the investigation of 2-Bit devices we add two additional levels using proportional conductance mapping [37] at 6.78 kΩ (=147.4 μS) for a ‘2’ and 12.74 kΩ (=78.47 μS) for a ‘1’. This leads to a HRS/LRS ratio of around 20. For this case, Figure 3(a) shows the number of generated pulses as a function of the number of devices in the HRS and for different conversion times. In this simulation, all WL inputs are at V_DD. A higher number of HRS represents smaller numerical values stored in the matrix and results in a larger number of generated pulses, as the smaller crossbar current (I_BL) is mirrored from N₀ to N₁ and results in a smaller voltage drop over R_Supp. This in turn increases the supply voltage of the VCO (V_{DD, VCO}) which increases its oscillation frequency. If the conversion time is increased, the number of generated pulses is increased linearly. This makes distinguishing the different levels easier, as the differences between adjacent levels are increased linearly as well. The black solid lines at the right side of Figure 3(a) indicate the required counter length which increases with increasing conversion times. Therefore, there exists a tradeoff between accuracy and variability tolerance on the one side and energy consumption and area on the other side, as longer counters consume more energy and require more space, but enable longer conversion times. Figure 3(b) shows the minimum difference between the number of generated pulses over the conversion time for the same inputs as in (a). As expected from (a), the difference increases very linearly from two at 160 ns conversion time to 12 for a conversion time of 760 ns. The conversion time consists of a setup time until the VCO reaches its correct frequency and the actual counting time. Based on the results from Figure 3 we can see how the ADC can trade off between low precision, low energy consumption and small area towards high precision, high energy consumption and large area. In this way, the operational parameters and design choices are made to follow different optimization routes depending, for example, on the device technology, the timing, the energy or the accuracy requirements of the application. From these initial simulations we define our baseline case as a conversion time of 260 ns for a minimum guaranteed pulse difference of four. Apart from being able to tolerate more device variability, the guaranteed minimum pulse difference also allows the ADC to distinguish between more dot products. Those dot products can be realized by using multilevel devices with additional conductance states between the minimum and maximum values. It should be noted, that the worst case for 1-Bit devices is observed for the dot products (V_DD 0 … 0) ⋅ (LRS HRS … HRS)^T = 1 and (V_DD V_DD … V_DD) ⋅ (HRS HRS … HRS)^T = 0. In this case the ADC has to differentiate between 16 selected HRS states (=95.92 kΩ/16 ≈ 6 kΩ) and one selected LRS (4.65 kΩ). For this case, the ADC produces a difference in the number of generated pulses of one for the baseline case (260 ns, 16 binary devices). To guarantee a perfect precision, this case needs to be distinguished. However, this worst case will occur with a very low probability. To mitigate this difficulty, higher HRS/LRS are required or less devices can be read out at the same time.

Figure 3:

Generated number of pulses as a function of the number of devices in the HRS for different conversion times (a). The required counter length is indicated on the right side of (a). (b) Shows the minimum difference in generated pulses for different conversion times.

3.3 Comparing different bit slicing cases

In our analysis we compared two different types of bit slicing, namely weight slicing between 1-Bit and 2-Bit devices and ADC slicing, where either 16, 8 or 4 rows are read out at the same time. The 1-Bit and 16 rows case was previously discussed as the baseline case for a conversion time of 260 ns, giving a guaranteed pulse difference of four in Figure 3. The three other bit slicing cases are then simulated for conversion times long enough to guarantee the same pulse difference, i.e. the same accuracy or noise resilience. In the case of 2-Bit devices this requires the counter to run for a longer time than in the case of 1-Bit devices, as the additional levels are added in between the 1-Bit levels. The longer run time of the counter can also require a bigger counter circuit, as indicated in Figure 3(a). Both factors increase the energy consumption, but a higher number of performed computations compensates them. In the case of using only 8 rows or 4 rows, the counter can be implemented smaller, as the maximum number of pulses to be counted is smaller. However, to achieve the same number of bit operations, the computation of 8 rows has to be repeated twice. For 4 rows it must be repeated four times. Under the requirement of the same precision, the conversion time is increased to 760 ns for the 2-Bit 16 rows case and it is decreased to 130 ns for 1-Bit/8 rows and decreased to 65 ns for 1-Bit/4 rows. The counter width required is 8-Bit in the baseline case, 9-Bit in the 2-Bit/16 rows case, 7-Bit in the 1-Bit/8 rows case and 5-Bit in the 1-Bit weight/4 rows case. The average energy consumption of the different components and the average total energy per bit are shown in Figure 4. For the total energy per bit the energies of the three components are added together and then divided by the number of bit operations. This number is 32 for 2-Bit 16 rows, 16 for 1-Bit 16 rows, 8 for 1-Bit 8 rows and 4 for 1-Bit 4 rows. From the results it can be seen, that the counter and select circuits are the largest energy consumer. As the select circuits are static during the dot product evaluation this energy is mostly consumed by the counter. The VCO and sensing stage are the second largest consumer and lastly the crossbar consumes the least amount of energy. Additionally, the results suggest that it is more energy efficient in the given architecture to perform many small dot product operations with 1-Bit devices, as those can be finished in a much faster time and with smaller counters.

Figure 4:

Average energy consumption of the different circuit components for the different bit slicing approaches.

3.4 Variability tolerance

VCM devices show different types of variability, such as device-to-device (d2d) variability, cycle-to-cycle (c2c) variability and read-to-read variability [38]. For the considered use case of VMM for machine learning accelerators, the most important variability effects are read disturb, read noise and programming variability [24]. Regarding the impact of read disturb, we showed in [24] that reading in the RESET direction is preferable to the SET direction and that read voltages up to 0.5 V over single devices are possible in the case of binary devices. As the read voltage here is 0.2 V in the RESET direction, read disturb will likely not be an issue. Regarding the read noise, we calculated it at the different conductance levels. Table 2 summarizes the VCM device conductances of the four considered levels and the minimum, median and maximum values for the amount of read noise and for the number of oxygen vacancies at these levels. The minimum and maximum numbers are based on the extreme cases for the filament volumes as will be described below in more detail. With the chosen transistor dimensions (W = 10 μm, L = 500 nm) the conductance of the 1T1R structure is almost completely determined by the resistances of the VCM cells. The JART VCM v1b Readvar model from [26] is then used to simulate the minimum, median and maximum amount of ΔG/G at the four different conductance levels, under the same read conditions as described above. The read variability model first discretizes the number of oxygen vacancies in the disc and then changes this number as described in [26]. As mentioned above, the filamentary oxide in the JART VCM v1b model consists of the plug region and the disc region, where the oxygen vacancy concentration in the disc is modulated during switching. For the discretization, the disc volume is multiplied with the oxygen vacancy concentration at the different levels. The worst case noise at a given vacancy concentration is then obtained for the smallest filament volume, as this gives the smallest number of oxygen vacancies. Adding or removing one vacancy has then a larger influence on the conductance, compared to larger disc volumes. The rightmost column in Table 2 shows the floored number of vacancies at the different conductance levels for the different filament volumes. The minimum number of vacancies leads to the maximum ΔG/G. As can be seen from the results, the worst-case noise can be very significant at the lower conductance levels (up to 1). The number of vacancies in this case is only two. However, most devices will have filament volumes close to the median case, where the read noise is between 5 % at the smallest conductance and 0.8 % at the highest conductance level. This read noise levels are then the best case accuracies for the different levels for the median devices. To program multilevel devices usually write verify algorithms are used [31, 39]. Even if it was possible to program levels with a 100 % accuracy, the intrinsic read noise would still lead to some uncertainty at the different conductance levels. With the read noise results we can reconsider the 1-Bit worst case from Section 3.1, between 16 selected HRS or level 0 states (=16·10.44 μS = 167.04 μS) and one selected LRS or level 3 (223.1 μS). In the JART VCM v1b Readvar model the read noise module can change the number of vacancies by two at most. In the worst case, all the devices at level zero gain two additional vacancies, increasing their conductance, while the single level 3 device has two vacancies removed, decreasing its conductance. In this case, the equivalent conductance will be increased by 10 % for the level 0 devices and it will be decreased by 1.6 % for the level 3 device. The two conductance levels are then 183.74 μS and 219.53 μS. With these worst case conductances considering read noise we can calculate the programming accuracy at which the two levels will overlap, making a distinction impossible. This is calculated by 219.53 μS/183.74 μS = 1.19, meaning that the program verify algorithms have to be able to achieve errors smaller than 9.5 %. With the given ADC two neighboring conductance states can in principle always be resolved, if the conversion time is increased. In practice, again it would be a better solution to use higher conductance ratios, or to relax the accuracy requirements to allow for some errors in the VMM. With regard to the bit slicing approaches higher device resolutions make the differentiation much more difficult, as the levels are here inserted between the 1-Bit levels. Then the conductances have to be spread out over a larger range or fewer rows have to be read out at the same time. Reading out fewer rows at a time does not directly affect the accuracy, due to the linear conductance spacing. However, it improves the energy consumption as shown in Section 3.3.