Multi person Detection

[mrussell246]

Enables detection and tracking of multiple people with DeepHRnet.

Makes use of SORT (Simple, Online and Realtime Tracker) algorithm, described here https://github.com/abewley/sort. The algorithm is placed in a file called sort.py in InfiniteSkyAI/pose_estimation/src/pose_estimation. SORT uses past and current frames to identify detections and assign them a label across frames. This tracker takes in the bounding boxes and confidences of found people, and returns a list of the bounding boxes with an associated ID number. This ID number should stay consistent and associate to specific people across frames. Via the max_age argument when instantiating the tracker, we can specify the maximum amount of frames we want to keep a detection alive in the case that there is a frame or series of frames with no detections for that person.

This tracker is used to identify the detected humans in the video. In order to return the keypoints for just the thrower, we use the logic that they are most likely closer to the camera, and therefore have a bigger bounding box area. We then find the ID number of the person with the biggest bounding box in the first frame, and use that ID number in subsequent frames to consistently return the correct keypoints.

With this method for identification, we could change criteria in the future for identifying a thrower, or return multi-person, sorted keypoints when needed.

In order to test, check out this branch as well as the multi_person_detection branch in pose_estimation. Then, simply run end_to_end and inspect the output DeepHRnet keypoint video to verify that the correct person was picked, and that the entire pipeline runs.

[John-HarringtonNZ]

What would be the correct way to test this addition? Is there a script to use or something else? @mrussell246

[John-HarringtonNZ]

From @mrussell246 about testing these changes:

"... The two branches you need are multi_person_detection in the deepHRnet repo and in pose_estimation, and the command to run to make sure everything works would just be end_to_end, which I ran on multiple videos. It took me a while since DeepHRNet takes a while to run..."