Artificial intelligence news.
Straight from the original announcements.

Action Recognition Beats Tracking for Small, Fast Video Movements

November 18, 2026, 23:22By the AI News Bites Editorial TeamSource: Roboflow
Action Recognition Beats Tracking for Small, Fast Video Movements
Image: Roboflow

Action recognition performed best for fast, subtle movements in Roboflow’s tests, while detection and tracking, or pretrained keypoints, were better suited to visible objects and body poses. The comparison covered an F1 pit stop, basketball and workout repetitions using held-out video clips.

Key points
  • Action recognition scored 62 in the F1 pit stop test, versus 49 for detection.
  • Action recognition caught all 5 tested basketball dribbles.
  • Keypoints and tracking scored 96 for workout repetitions.
  • The pit-stop action model improved after sampling video at 8 fps.
  • Roboflow recommends choosing the method based on the video question.

Three tasks, three different winners

Roboflow built action recognition and tracking-based systems for the same three tasks, then evaluated them on held-out clips. The action recognition models analyzed video clips to identify what happened and when. The alternative used object detection or keypoints, tracking and rules.

Action recognition won the F1 pit stop test with a segment F1 score of 62, compared with 49 for detection. It also led in basketball, scoring 56 versus 38 for detection. For workout repetitions, keypoints and tracking scored 96, compared with 56 for action recognition.

Roboflow says the scores range from 0 to 100. A prediction counted when it had the correct label and overlapped the real action by at least half.

Why small movements favored action recognition

The F1 pit stop takes about three seconds and includes six steps. The jack lifts the car a few centimeters in about a quarter of a second, while crew members can block its view. The detection and tracking system did not catch a single jack step because the car’s box barely moved and the jack was obscured.

The first action recognition version also missed the movement because it sampled video at 4 frames per second. Roboflow retrained the same model at 8 fps, changing nothing else, and the updated system became the best pit stop system overall. Tracking remained more effective for visible events such as checking whether the original tire was still on the car.

Basketball showed a similar pattern. A rule-based system caught 1 of 5 dribbles, while action recognition caught all 5. The action model also learned made shots from 13 training examples, without needing to locate the ball or rim during the entire event.

Keypoints handled exercise repetitions

For workouts, Roboflow used a pretrained RF-DETR keypoint model without fine-tuning. It provided 17 points per person, allowing rules to identify exercises from body geometry. A push-up was defined by the torso tilting more than 40° from vertical, while a jumping jack was identified when the wrists rose above the shoulders. Squats and lunges were counted when the hips dipped, with different knee heights distinguishing a lunge.

These rules counted every repetition with zero training. Action recognition was nearly perfect at identifying the exercise, but merged ten jumping jacks into one long segment, so it could not count repetitions by itself. Roboflow recommends using keypoints for questions about body pose, tracking for visible objects and locations, and action recognition for movements that unfold across several frames.

For action recognition, the company advises labeling dozens of clips per action, increasing the video sampling rate for actions lasting a second or less, and adding a counter when repetitions are needed. Its tests also suggest that a practical system may combine methods, such as tracking tires while using action recognition for pit-stop jack movements.

Related news

CoreWeave AI cuts Aston Martin F1 aerodynamic prediction error by 50%
ResearchNovember 16, 2026, 23:53

CoreWeave AI cuts Aston Martin F1 aerodynamic prediction error by 50%

CoreWeave and the Aston Martin Aramco Formula One Team have built a machine learning model that reduces the error in predicting on-track aerodynamic performance from…

Roboflow Adds Video Mirroring from S3, GCS and Azure
Products and toolsNovember 12, 2026, 20:57

Roboflow Adds Video Mirroring from S3, GCS and Azure

Roboflow has added video mirroring from AWS S3, S3-compatible storage, Google Cloud Storage and Azure Blob Storage to its Datasources feature. Teams can connect cloud…

Roboflow Adds Action Recognition for Video-Based Workflow Analysis
Products and toolsNovember 11, 2026, 14:19

Roboflow Adds Action Recognition for Video-Based Workflow Analysis

Roboflow has added support for action recognition, enabling teams to train and deploy vision models that identify actions across video and return a timeline with start…