A pointer location algorithm for computer visionbased automatic reading recognition of pointer gauges

Erlin Tian; Huanlong Zhang; Marlia Mohd Hanafiah

doi:10.1515/phys-2019-0010

Artikel Open Access

A pointer location algorithm for computer visionbased automatic reading recognition of pointer gauges

Erlin Tian , Huanlong Zhang und Marlia Mohd Hanafiah

Veröffentlicht/Copyright: 5. April 2019

Veröffentlicht von

Veröffentlichen auch Sie bei De Gruyter Brill

Manuskript einreichen Informationen für Autor*innen Erkunden Sie dieses Fachgebiet

Aus der Zeitschrift Open Physics Band 17 Heft 1

Abstract

In order to implement a computer vision based reading recognition system for pointer gauges, we propose an algorithm to locate the pointer by searching for the minimal bounding rectangle (MBR) of the pointer region. Having determined the center of dial region and the centroids of scale marks, we build up a reference system for reading recognition with the dial region center and the scale mark centroids. To identify the indicating value of a gauge image, we first detect the pointer region based on frame difference, and determine the MBR of the pointer using the longest edge of its convex hull. We then take the centroid of the MBR enclosing the pointer region as the centroid of the pointer. We next take the connection between the pointer centroid to the dial center as the pointer, and calculate the indicating value corresponding to the pointer based on the angle method. Experimental results reveal the effectiveness of the pointer location algorithm in computer vision based reading recognition for pointer gauges.

Keywords: Computer vision; pointer location; reading recognition; convex hull

PACS: 07.05.-t; 07.05.Kf; 07.05.Rm

1 Introduction

Even though digital instruments and meters have become more and more popular, gauges with pointers are still widely used because of all kinds of advantages [1, 2, 3]. However, digital output signals are not available when using pointer gauges, whose measurement results have to be obtained through manual reading. To some extent, the inefficiency and unreliability caused by manual reading reduce the performance of conventional metrological verification systems for gauges with pointers. Therefore, machine vision technologies which use cameras as human eyes have been adopted to automatically read the measurement results of pointer gauges. The machine vision methods are particularly applicable to this area under the background that the computer performance has been dramatically improved. Machine vision-based automatic reading recognition technology has always been attracting attention because it is about specific applications. Due to the variety of shape and appearance of the pointer meter, there is no universal solution that can achieve good results under various conditions. Nevertheless, the key technologies that influence the performance of the recognition system include scale marks detection, pointer detection and reading recognition. This is similar to human observation.

For scale marks detection, the connected-component method [4] and the central projection method [5] are most often adopted after scale region segmentation. For pointer detection, several methods like frame difference [6], Hough line detection [7, 8], straight-line fitting [9], template matching [10] and central projection [11] have been proposed. There are mainly two methods for reading recognition when the pointer has been detected. One is the angle method [12], which intends to calculate the deflection angle between the target pointer and the scale marks. The other is the distance method [1], which recognizes the indicating value according to the distance between the detected pointer and corresponding scale marks [13, 14, 15, 16, 17].

As shown in Figure 1, the background of the scale region comes in three colors in our application. To tackle this problem, we locate the scale region in the S component of the HSV color space based gauge image, and segment this region from the V component. Having detected the scale marks using the connected-component method, we select to use the frame difference method to detect the pointer, and use the MBR method to locate the centroid of the pointer. More importantly, we propose a fast algorithm to determine the MBR of the pointer using the longest edge of its convex hull. Finally, we select to use the angle method to obtain the indicating value.

Figure 1

An example image of a pointer gauge

In the rest of the paper, we will introduce the overall architecture of the whole framework for reading recognition in Section 2. Then we will describe the details of gauge pointer detection for reading recognition in the next section. We will present the experiments and corresponding results in Section 4 and conclude the paper in Section 5.

2 Overview of the framework for reading recognition

As seen in Figure 2, the whole reading recognition scheme adopted in this paper can be divided into 4 steps including dial template location, scale marks detection, pointer detection and reading recognition.

2.1 Dial template location

For each gauge, two gauge images in which the pointers are at different positions are first selected. Since the pixels in the background of the scale region have larger pixel values, the reference image of the dial template without pointers can be obtained by comparing the two images pixel by pixel and taking the larger value at each pixel. In this way, the Hough circle detection method can be then used to determine the position of the dial by searching for the center of the dial in the V component of the reference image.

2.2 Scale marks detection

The scale region in the S component of the dial template is a region with high color saturation. Accordingly, we can utilize image processing methods to locate the scale region in the S component of the dial template. The S component of the dial template is first converted to a binary image using a fixed threshold value. Having determined the center and radius of the dial, we can segment the annular region with colored background from the dial template. The scale region is finally accurately located by excluding those connected-components with small areas. The scale region can be then segmented from the V component of the dial template since its location has been determined. After binarization, connected-component labeling is adopted to locate the centroid of each scale mark.

2.3 Pointer detection

The pointer of each gauge image to be recognized is detected using a frame difference method based on the gauge image and the reference image of the dial template. Then the pointer region is located using its MBR, and the centroid of the MBR can be taken as the centroid of the pointer. Therefore, the connection between the center of the dial and the centroid of the MBR can be seen as the pointer. The details of the process of pointer detection will be given in Section 3.

2.4 Reading recognition

Having determined the positions of the scale marks, we can recognize the indicating value of a pointer based on the angle method. We first connect the centroid of each scale mark to the center of the dial. We then connect the centroid of the pointer to the dial center and calculate the deflection angle between this connection and each connection corresponding to each scale mark. The closest scale mark is then taken as a reference for reading recognition. To calculate the deflection angle between the pointer and each scale mark, we build up a rectangular coordinate system whose origin is the dial center. Given the centroid of a pointer, we connect it to the dial center and calculate the deflection angle between the reference line and the connection. The scale mark corresponding to the minimal deflection angle is finally selected since it is closest to the pointer. The reading result of a pointer can be calculated based on the fact that the scale marks are evenly distributed.

Figure 2

The overall architecture of the proposed framework for reading recognition

3 Pointer detection and location

Given a gauge image corresponding to an indicating value, we convert it into HSV space and obtain its V component. To avoid the interference of background area and the center hole of the dial, we segment the annular scale region of the dial from the V component. As seen in Figure 3, this segmentation is also done in the V component of the dial template. We then subtract the segmented V component of the gauge image from that of the dial template. After binarization and closing operation [18], the pointer can be detected. The centroid of the pointer region is then determined and connected to the center of the dial to represent the pointer.

However, the detected pointer region may be incomplete when the pointer overlaps with the scale marks and characters. The closing operation has little effect at this time. In this case, the centroid of the pointer will deviate if we determine it by taking the average coordinates of the pixels in the pointer region as the coordinates of its centroid. Considering the shape of the pointer is approximately rectangular, we can use the MBR to enclose the pointer region. Since the MBR is approximate to the ideal pointer, the centroid of the MBR can then be taken as the centroid of the pointer.

However, it is computationally expensive to search for the MBR. There is a commonly used method to find the minimal rectangle based on a technique called “rotating calipers” [19, 20, 21]. This method depends on the fact that one side of the minimal bounding rectangle of an object coincides with one edge of the convex polygon containing this object. Therefore, only the orientations of the convex polygon’s edges have to be considered in the process of searching for the MBR. An efficient “rotating calipers” algorithm [22] to determine the MBR of an object is shown in Figure 4. The convex hull of the object is first obtained and every edge of the convex hull is checked by spreading a pair of calipers around the object at the angle of each edge. The edge producing the minimum area from all the possible rectangles is finally chosen and the corresponding MBR is determined.

In our application, the pointer is thin. The longest edge of the convex polygon containing the pointer region can roughly determine the orientation of the longer side of the minimal rectangle enclosing the pointer. So we use a simplified “rotating calipers” algorithm without rotating the convex hull in every direction of its edges. As shown in Figure 5, the detailed process is as follows:

(1) For a detected pointer region, its convex hull is first computed based on the boundary points.

(2) The longest edge of the convex hull is selected. Suppose the two end-points of the longest edge are (x₁, y₁) and (x₂, y₂), the angle between this edge and the horizontal axis can be calculated as:

(1) θ = − arctan y 2 − y 1 x 2 − x 1

Figure 3

The process of pointer detection

Figure 4

The conventional “rotating calipers” method to determine the MBR of an object

Here the minus sign is used because of the fact that the y-coordinate increases downward.

(3) The vertices of the convex hull are then rotated clockwise by an angle of θ to make the convex hull’s longest edge parallel to the horizontal axis. The coordinate transformation can be expressed as [23]

(2) ( s , t ) = ( x , y ) cos ⁡ θ sin ⁡ θ − sin ⁡ θ cos ⁡ θ

Figure 5

The determination process of MBR based on the longest edge of the convex hull

Here (x, y) denotes the original coordinates of a point and (s, t) denotes the coordinates after rotation. If the angle of θ is negative, the convex hull is rotated anticlockwise.

(4) The minimum and maximum coordinate values of the vertices after rotation can be determined as: s_min, s_max, t_min, t_max. As seen in the subfigure (d) of Figure 5, the four points (s_min, t_min), (s_max, t_min), (s_max, t_max) and (s_min, t_max) can determine a rectangle enclosing the pointer. This rectangle is then taken as the MBR. The coordinates of the MBR’s centroid can be calculated as:

(3) s c = s min + ( s max − s min ) / 2

(4) t c = t min + ( t max − t min ) / 2

(5) The rectangle is then rotated by an angle of –θ to the original location of the pointer. As seen in the subfigure (e) of Figure 5, the MBR is able to label the complete pointer region. The centroid of the rotated MBR is located as:

(5) ( x c , y c ) = ( s c , t c ) cos ⁡ θ − sin ⁡ θ sin ⁡ θ cos ⁡ θ

This point can be taken as the pointer’s centroid in order to correct the centroid deviation of an incompletely detected pointer. It is obtained by rotating the convex hull only once.

4 Experiments

In order to evaluate the performance of the MBR based method we propose for locating the centroid of a pointer, we used a standard gauge to verify the reading recognition performance. The indicating value of the gauge ranges from −0.1 to 0.9 and the minimal interval value between scale marks is 0.02. There are 51 scale marks in the scale region. As shown in Figure 6, we selected 10 test values and captured 10 gauge images of size 531 × 800 corresponding to these test values. Here the V components of the 10 gauge images are fused into one image to save space.

Figure 7 gives the result of pointer detection using the proposed method. The detection results of the 10 pointers are also fused into one image. The detected MBRs used to enclose the pointers are highlighted in red. The blue stars stand for the centroids of the MBRs enclosing the pointers. As seen in Figure 7, our proposed method can accurately label the pointers. To provide a quantitative presentation, we give the centroids’ coordinates of the detected MBRs in Table 1. This indicates the effectiveness of the method we propose to determine the MBR of a pointer from another perspective.

Table 1

The centroids’ coordinates of the detected MBRs and the time consumption of MBR determination

No.	Our method		Time
	x	y	consumption (ms)
1	272.5	293.4	0.76
2	278.9	240.5	0.73
3	307.3	188.3	0.79
4	353.8	155.6	0.68
5	405.5	144.0	0.88
6	463.5	155.5	0.70
7	509.1	189.0	0.73
8	537.5	236.1	0.79
9	544.4	289.8	0.74
10	530.9	339.6	0.74

Figure 6

The fused image representing the V components of the 10 gauge images

In the process of pointer detection, the MBR of a pointer is determined according to the longest edge of its convex hull. In theory, this method is efficient since it avoids rotating the convex hull many times. To evaluate the efficiency, we also give the average time consumption for locating the MBR of each pointer using the proposed method in Table 1. The results were obtained by running the same test 10 times. Experiments reveal that the time consumption of the method we propose is less than 1ms. This indicates that our method can be successfully implemented with low time consumption, which shows the good real-time performance of the proposed method.

Figure 7

The result of pointer detection

To compare with the results of manual reading, we further give the recognition results of the standard gauge in Table 2 and Figure 8. It can be seen that the MBR based method can achieve higher accuracy. What is more, the recognition results of the method we propose are nearly the same to that of the manual reading method which determines the indicating values by human eyes. If we use the results of the manual reading method as reference values, the maximum relative error of our method is only 0.67%. This further shows the effectiveness of the method we propose.

Table 2

The recognition results of a standard gauge

No.	Manual reading (MPa)	Our method (MPa)	Relative error compared to the manual reading method (%)
1	0.055	0.0549	0.18
2	0.140	0.1396	0.29
3	0.235	0.2337	0.55
4	0.324	0.3230	0.31
5	0.407	0.4057	0.32
6	0.500	0.4977	0.46
7	0.585	0.5856	0.10
8	0.672	0.6710	0.15
9	0.760	0.7549	0.67
10	0.840	0.8352	0.57

Figure 8

The recognition results of a standard gauge

5 Conclusion

Aiming at a machine vision based reading recognition system for an analogue gauge with pointers, we propose an algorithm for pointer detection by enclosing the pointers with MBRs to solve the problem of centroid deviation for incompletely detected pointers. Experimental results reveal that the method we propose for pointer detection can locate a pointer accurately and efficiently by determining its MBR according to the longest edge of its convex hull. This guarantees the efficiency and effectiveness of the overall scheme for reading recognition. Even though the pointer detection method is proposed for a specific application, it can still provide a valuable reference for similar applications concerning reading recognition. What is more, the algorithm for determining the MBR of an object based on the longest edge of the convex hull can be even applied to other related areas.

Acknowledgement

This work was supported by the National Natural Science Foundation of China under Grant 61873246, and the Scientific and Technological Project of Henan Province under Grant 182102210607 and 192102210108.

Conflict of Interest

Conflict of Interests: The authors declare that there is no conflict of interests regarding the publication of this paper.

References

[1] Chi J., Liu L., Liu J., Jiang Z., Zhang G.Machine vision based automatic detection method of indicating values of a pointer gauge, Math. Prob. Eng., 2015, (2015-10-4), 2015, 2015(3), 1-19.10.1155/2015/283629Suche in Google Scholar

[2] Belan P.A., Araujo S.A., Librantz A.F.H., Segmentation-free approaches of computer vision for automatic calibration of digital and analog instruments, Meas. J. Int. Meas. Conf., 2013, 46(1), 177-184.10.1016/j.measurement.2012.06.005Suche in Google Scholar

[3] Alegria E.C., Serra A.C. Automatic calibration of analog and digital measuring instruments using computer vision, IEEE Trans. Instrum. Measurement, 2000, 49(1), 94-99.10.1109/19.836317Suche in Google Scholar

[4] Zhang Y., Zhang D., Xu C.H., Xie B., Imagecalibration algorithm based on total least squares method for pointer instruments, Proc. Autom. Instrum., 2015, 36(5), 75-79.Suche in Google Scholar

[5] Luo D.C., Wang S.C., Zeng H.G., Zhao-Zao L.I., Liu X.M. Design of recognition system of analog measuring instruments, Laser Infr., 2007, 37(4), 377-380.Suche in Google Scholar

[6] Zhang J.,Wang J.H., Chi-Ming Y.U., Ding M.Y., Hou W.G. Novel automobile meter pointer detection algorithm based on computer vision, Comp. Eng. Sci., 2013, 35(3), 134-139.Suche in Google Scholar

[7] Tao B.J., Han J.L., Li E., An applied method for reading recognition of index-instrument, Opto-Electr. Eng., 2011, 38(04), 145-150.Suche in Google Scholar

[8] Xu L., Fang T., Gao X. An automatic recognition method of pointer instrument based on improved hough transform, Proceedings of AOPC 2015: Image Processing and Analysis, International Society for Optics and Photonics, 2015, 9675, 96752T.10.1117/12.2202805Suche in Google Scholar

[9] Wang Q., Fang Y.,Wang W.,Wu M., Wang R., Fang Y. Research on automatic reading recognition of index instruments based on computer vision, Proceedings of 2013 3rd International Conference on Computer Science and Network Technology, IEEE, 2013, 10-13.10.1109/ICCSNT.2013.6967053Suche in Google Scholar

[10] Yang Z., Niu W., Peng X., Gao Y., Qiao Y., Dai Y. An image-based intelligent system for pointer instrument reading, Proceedings of 2014 4th IEEE International Conference on Information Science and Technology, IEEE, 2014, 780-783.10.1109/ICIST.2014.6920593Suche in Google Scholar

[11] Gellaboina M.K., Swaminathan G., Venkoparao V. Analog dial gauge reader for handheld devices, Proceedings of 2013 IEEE 8th Conference on Industrial Electronics and Applications (ICIEA), IEEE, 2013: 1147-1150.10.1109/ICIEA.2013.6566539Suche in Google Scholar

[12] Wang R., Qi L.I., Fang Y.J. An automatic reading method of pointer instrument’s images based on improved angle method, Electrical Measurement and Instrumentation, 2013.Suche in Google Scholar

[13] Yang A., Han Y., Pan Y., Xing H., Li J. Optimum surface roughness prediction for titanium alloy by adopting response surface methodology, Res. Phys., 2017, 7, 1046-1050.10.1016/j.rinp.2017.02.027Suche in Google Scholar

[14] Plata S.A., Sáez S.B., After notes on Chebyshev’s iterative method, Appl. Math. Nonlin. Sci., 2017, 2(3), 1-12.10.21042/AMNS.2017.1.00001Suche in Google Scholar

[15] Duan M., Liu Z., Yan D., Peng W., Baghban A. Application of Lssvm algorithm for estimating higher heating value of biomass based on ultimate analysis, Ener. Sourc. Part A: Recov. Util. Envir. Eff., 2018, 40(6), 709-715.10.1080/15567036.2018.1454552Suche in Google Scholar

[16] Zong H., Cao Y., Liu Z. Energy security in group of seven (g7): A quantitative approach for renewable energy policy, Ener. Sourc. Part B: Econ. Plan. Policy, 2018, 13(3), 173-175.10.1080/15567249.2017.1422053Suche in Google Scholar

[17] Li D., Wang L., Peng W., Ge S., Li N., Furuta Y. Chemical structure of hemicellulosic polymers isolated from bamboo bio-composite during mold pressing, Polym. Compos., 2017, 38(9), 2009-2015.10.1002/pc.23772Suche in Google Scholar

[18] Lei T., Jia X., Zhang Y., He L., Meng H., Nandi A.K. Significantly fast and robust fuzzy c-means clustering algorithm based on morphological reconstruction and membership filtering, IEEE Trans. Fuzzy Syst., 2018, (99), 1.10.1109/TFUZZ.2018.2796074Suche in Google Scholar

[19] Pirzadeh H., Computational geometry with the rotating calipers,1999, Master’S Thesis McGill Univ.Suche in Google Scholar

[20] Baig A.Q., Naeem M., Gao W. Revan and hyper-revan indices of octahedral and icosahedral networks, Appl. Math. Nonlin. Sci., 2018, 3(1), 33-40.10.21042/AMNS.2018.1.00004Suche in Google Scholar

[21] Gao W., Wang W. A Tight neighborhood union condition on fractional (G, F, N ’, M)-critical deleted graphs, Coll. Math., 2017, 149(2), 291-298.10.4064/cm6959-8-2016Suche in Google Scholar

[22] D’Errico J., A suite of minimal bounding objects, http://www.mathworks.com/matlabcentral/fileexchange/34767-a-suite-of-minimal-bounding-objects, May 2014.Suche in Google Scholar

[23] Yarman C.E., Yazõcõ B., A new exact inversion method for exponential radon transform using the harmonic analysis of the euclidean motion group, Inv. Probl. Imag., 2017, 1(3), 457-479.10.3934/ipi.2007.1.457Suche in Google Scholar

Received: 2018-10-30

Accepted: 2019-01-28

Published Online: 2019-04-05

This work is licensed under the Creative Commons Attribution 4.0 International License.

Artikel in diesem Heft

https://doi.org/10.1515/phys-2019-0010

Schlagwörter für diesen Artikel

Computer vision; pointer location; reading recognition; convex hull

Creative Commons

BY 4.0