如何使用opencv检测手指数并添加两位数？

Question

我是新的OpenCV和图像处理。我正在尝试创建一个程序，检测显示在相机前面的手指数量，并添加它们。 在GitHub上搜索了一段时间后，我发现了一个项目，它可以检测一只手并计算其中的手指数。你知道吗

我在执行过程中面临的问题是，除非我的背景清楚，否则它不会给我显示数字。它一直在改变数字。我怎样才能稳定下来？因为我需要将屏幕前显示的数字相加，所以除非数字稳定，否则我无法存储它。你知道吗

import traceback
import cv2
import numpy as np
import math


cap = cv2.VideoCapture(0)

while(1):

    try:  #an error comes if it does not find anything in window as it cannot find contour of max area
          #therefore this try error statement

        ret, frame = cap.read()
        frame=cv2.flip(frame,1)
        kernel = np.ones((3,3),np.uint8)

        #define region of interest
        roi=frame[100:300, 100:300]


        cv2.rectangle(frame,(100,100),(300,300),(0,255,0),0)    
        hsv = cv2.cvtColor(roi, cv2.COLOR_BGR2HSV)



    # define range of skin color in HSV
        lower_skin = np.array([0,20,70], dtype=np.uint8)
        upper_skin = np.array([20,255,255], dtype=np.uint8)

     #extract skin colur imagw  
        mask = cv2.inRange(hsv, lower_skin, upper_skin)



    #extrapolate the hand to fill dark spots within
        mask = cv2.dilate(mask,kernel,iterations = 4)

    #blur the image
        mask = cv2.GaussianBlur(mask,(5,5),100) 

    #find contours
        contours,hierarchy= cv2.findContours(mask,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
        print(contours)
        print(hierarchy)
   #find contour of max area(hand)
        cnt = max(contours, key = lambda x: cv2.contourArea(x))

    #approx the contour a little
        epsilon = 0.0005*cv2.arcLength(cnt,True)
        approx= cv2.approxPolyDP(cnt,epsilon,True)


    #make convex hull around hand
        hull = cv2.convexHull(cnt)

     #define area of hull and area of hand
        areahull = cv2.contourArea(hull)
        areacnt = cv2.contourArea(cnt)

    #find the percentage of area not covered by hand in convex hull
        arearatio=((areahull-areacnt)/areacnt)*100

     #find the defects in convex hull with respect to hand
        hull = cv2.convexHull(approx, returnPoints=False)
        defects = cv2.convexityDefects(approx, hull)

    # l = no. of defects
        l=0

    #code for finding no. of defects due to fingers
        for i in range(defects.shape[0]):
            s,e,f,d = defects[i,0]
            start = tuple(approx[s][0])
            end = tuple(approx[e][0])
            far = tuple(approx[f][0])
            pt= (100,180)


            # find length of all sides of triangle
            a = math.sqrt((end[0] - start[0])**2 + (end[1] - start[1])**2)
            b = math.sqrt((far[0] - start[0])**2 + (far[1] - start[1])**2)
            c = math.sqrt((end[0] - far[0])**2 + (end[1] - far[1])**2)
            s = (a+b+c)/2
            ar = math.sqrt(s*(s-a)*(s-b)*(s-c))

            #distance between point and convex hull
            d=(2*ar)/a

            # apply cosine rule here
            angle = math.acos((b**2 + c**2 - a**2)/(2*b*c)) * 57


            # ignore angles > 90 and ignore points very close to convex hull(they generally come due to noise)
            if angle <= 90 and d>30:
                l += 1
                cv2.circle(roi, far, 3, [255,0,0], -1)

            #draw lines around hand
            cv2.line(roi,start, end, [0,255,0], 2)


        l+=1

        #print corresponding gestures which are in their ranges
        font = cv2.FONT_HERSHEY_SIMPLEX
        if l==1:
            if areacnt<2000:
                cv2.putText(frame,'Put hand in the box',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
            else:
                if arearatio<12:
                    cv2.putText(frame,'0',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
                elif arearatio<17.5:
                    cv2.putText(frame,'Best of luck',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

                else:
                    cv2.putText(frame,'1',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        elif l==2:
            cv2.putText(frame,'2',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        elif l==3:

              if arearatio<27:
                    cv2.putText(frame,'3',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)
              else:
                    cv2.putText(frame,'ok',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        elif l==4:
            cv2.putText(frame,'4',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        elif l==5:
            cv2.putText(frame,'5',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        elif l==6:
            cv2.putText(frame,'reposition',(0,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        else :
            cv2.putText(frame,'reposition',(10,50), font, 2, (0,0,255), 3, cv2.LINE_AA)

        #show the windows
        cv2.imshow('mask',mask)
        cv2.imshow('frame',frame)
    except Exception:
        traceback.print_exc()
        pass
       # break


    k = cv2.waitKey(5) & 0xFF
    if k == 27:
        break

cv2.destroyAllWindows()
cap.release()

这是完整的代码。 我觉得肤色的HSV范围不准确。我是一个棕色的印度人，在面具屏幕上，它使皮肤的颜色范围从白色到1，其余的，我可以看到在我的手掌部分地区是黑色的。你知道吗

有没有办法通过保持时间限制来增加两位数？ 例如，当我显示5并且在1秒之内，如果我显示3，它应该将输入作为53。你知道吗

一个带有例子或示范的答案会很有帮助。 非常感谢。你知道吗

Answer 1

最常用的消除数据抖动的方法是收集更多的数据并从中获得平均值或最频繁的数据。你知道吗

你可以测量一秒钟，然后取其中出现最多的数字。例如，如果有10帧分别显示[5，3，1，5，5，4，5，4，3，2]个手指，则输出应该是5个手指。然后将手指数添加到先前测量的手指数中，在屏幕上显示并继续测量。你知道吗

首先也是最重要的一点，我认为这将是一个好主意，使一个方法，采取了一个图像，并返回了手指的数量测量。我们称之为DetectNrFingers（图像）。您应该能够很容易地从上面的所有代码中得到这一点，因为您知道哪个变量打印手指的数量。你知道吗

你要加两位数。这是另一个问题。如果你总是有两位数，这很简单，你可以测量4倍的总数，例如，5，3，4，2。那么答案是53+42。如果不是两位数的话，你可以让这个人先把0举到一位数之前，然后再测量4次。你知道吗

我通常在C++中工作，所以编写一个代码示例可能会出错。我用一种python风格编写了一些伪代码，希望您能理解。让我们伪代码：

//set the start time so you know how much time has passed after the first measurement
start_time = time.time() 

//this is how long you want to measure the fingers
timeToMeasure = 1000 

//initialize with a not possible number
previousAmountFingers = -1
prevCombinedFingers = -1

while(1):
  //Get the image Mat from the camera. It can be the raw feed, or it can be preprocessed depending on how you want to display it later.
  image = GetImageFromCamera() 

  listNrOfFingers.append(DetectNrFingers(image))

  elapsed_time = time.time() - start_time

  if(elapsed_time > timeToMeasure)
    //this is my example above, which would return a 5 in [5, 3, 1, 5, 5, 4, 5, 4, 3, 2]
    amountOfFingers = numberOccuringTheMost(listNrOfFingers) 
    listNrOfFingers.Clear()

    if(previousAmountFingers != -1) //the initial value
      //combine seperate integers to one, for example, 5 and 3 -> 53
      combinedFingers = int(str(amountOfFingers) + str(previousAmountFingers))
      previousAmountFingers = -1 

      if(prevCombinedFingers != -1)
        sumFingers = combinedFingers + prevCombinedFingers
       //putText sumFingers into the image on a position you like

      prevCombinedFingers = combinedFingers
    else //only if the previous digit was not initialized yet:
      previousAmountFingers = amountOfFingers
    start_time = time.time() //reset the timer for the next round of measurements.

因为你需要做实时，我得到，每帧1秒是不够的，但是你可以延迟你的计算一点，我猜。只需打印上一次计算的总和，直到下一次计算完毕。因此，即使您还没有计算手指，也要显示更多的帧。当然你可以将1秒设置为更少，但是你也需要时间来调整你的手，所以如果你想显示手本身，你需要显示更多的帧而不是答案。你知道吗