Technical Paper

Keywords Camera Tracking // Open Source Tools // Unreal Engine // Optic Flow

Camera Tracking Systems and their Democratisation

Irene Muñoz López, Andrew Gilbert

Centre for Creative Arts and Technologies (C-CATS), University of Surrey
SMPTE Motion Imaging Journal, January/February 2026. DOI: 10.5594/JMI.2026/9GPF670

Journal PDF

Tweet Share

Summary: This work analyses camera tracking systems used in virtual production, building an open-source OpenCV tracker and comparing it against industry-standard tools (Blender and After Effects). By varying video noise levels and resolutions, the paper evaluates performance, workflow, and the democratisation of tracking technology for the general public.

Builds a markerless OpenCV camera tracker to compete with commercial tools.
Studies the impact of Gaussian noise and resolution on tracking performance.
Assesses accessibility and workflow in Unreal Engine for qualitative evaluation.

Tracking Workflow Overview

The camera tracking workflow begins with detection of salient features, followed by execution of the tracking algorithm, extraction of translation and rotation vectors, and integration of motion over time to obtain camera coordinates and Euler angles.

Figure 1. Overview of the camera tracking study. The paper compares an OpenCV-based tracker with Blender and After Effects under different video noise levels and resolutions, integrating the resulting camera paths into Unreal Engine for qualitative assessment.

Abstract

Computer vision encompasses the analysis, processing, and interpretation of visual data. Tracking is a subset of this field, where systems recognise salient features in a scene to determine their displacement across frames in a video stream. This facilitates automation, increases efficiency, and broadens the functionality of these systems in applications such as surveillance, medicine, and entertainment.

Recent interest in Virtual Reality (VR) and Augmented Reality (AR) has prompted the development of new camera tracking techniques. Many tools are available to analyse and process tracking information, but most are proprietary and not accessible to the general public, limiting democratisation. This paper compares three camera tracking systems: two industry-standard tools (Blender and After Effects) and a tracker built using OpenCV’s open-source tools. By examining tracking accuracy under different video resolutions and Gaussian noise levels, and by importing tracks into Unreal Engine for qualitative assessment, the study evaluates workflow, optimisation, and democratisation of camera tracking technology.

Introduction

Computer vision systems trained to analyse digital images and video enhance workflow automation and efficiency. Reliable tracking of objects or salient features across frames is fundamental to surveillance, security, medicine, and human–computer interaction. Tracking techniques estimate the camera’s geometry and pose in a scene and are central to VR and AR applications, as well as virtual production.

Commercial camera tracking tools increasingly support markerless workflows, yet the cost and proprietary nature of these tools limit their accessibility. Open-source libraries, such as OpenCV, provide building blocks for bespoke tracking systems but require technical expertise. In this context, it is important to understand how far low-cost or free systems can compete with professional solutions and what constraints noise, resolution, and workflow impose on real-world usage.

The aim of this research is to understand the workflow, optimisation, and democratisation of common camera tracking systems by:

Creating an OpenCV-based camera tracker that can compete with industry-standard tools.
Investigating how different video resolutions and Gaussian noise levels affect tracking accuracy.
Evaluating qualitative performance in a virtual environment built in Unreal Engine.

Tracking Workflow Overview

The camera tracking workflow begins with detection of salient features, followed by execution of the tracking algorithm, extraction of translation and rotation vectors, and integration of motion over time to obtain camera coordinates and Euler angles.

Figure 1. Overview of a tracking system. Salient features are detected and tracked across frames, with extracted motion integrated over time to estimate camera movement.

Methodology

1. Building an OpenCV Camera Tracker

The OpenCV-based tracker detects salient features using SIFT and tracks their motion via Lucas–Kanade optic flow across the video. From the resulting motion vectors, the Essential matrix and camera matrix are used to estimate camera pose, with translation and rotation vectors integrated over time to obtain trajectories in $$x$$, $$y$$, and $$z$$. Euler angles describing roll, pitch, and yaw are extracted for each frame, forming a camera path that can be imported into Unreal Engine.

2. Experimental Video Dataset

Five 10‑second videos of a hand‑held walk along a path were recorded at $$4K$$ resolution (3840 × 2160). Each video was versioned by adding Gaussian noise at 25 %, 50 %, and 75 % of pixels and by encoding at multiple resolutions (1080p, 720p, 480p). These versions allowed a systematic study of how noise and resolution influence tracking performance for each system.

3. Comparative Tracking Systems

In addition to the OpenCV tracker, the study uses:

Blender – open-source 3D software with camera tracking tools based on feature detection and optic flow.
Adobe After Effects – proprietary compositing and tracking tool widely used in industry.

All systems estimate camera motion, and the resulting tracks are evaluated using quantitative metrics (mean error per frame and standard deviation) and qualitative inspection in Unreal Engine.

4. Integration into Unreal Engine

Camera paths extracted from each tracking system are transformed to Unreal Engine’s left‑handed coordinate system. A Python API is used to create camera actors and import coordinates as keyframes. Virtual scenes are constructed to match the physical environment, enabling side‑by‑side comparison between video footage and virtual renders.

Dataset and Experimental Setup

The paper illustrates salient feature detection in outdoor scenes, versions of the dataset with varying Gaussian noise and resolution, and examples of frames used for qualitative comparison.

Figure 2. Salient features detected using OpenCV’s SIFT feature detector, marked with red crosses.

Figure 3. Overview of the experimental workflow. Original videos are versioned with added Gaussian noise and different resolutions, tracked using Blender, After Effects, and the OpenCV tracker, and then imported into Unreal Engine for qualitative analyses.

Frames from one of the videos in the dataset

Figure 4. Frames from one of the dataset videos showing the hand‑held walk along a path used in the experiments.

Quantitative Results

Tracking performance is quantified using the mean error per frame and standard deviation of the tracked camera position relative to a reference track. Two main factors are investigated: Gaussian noise percentage and video resolution.

Impact of Noise on Tracking Performance

Tracker	% Gaussian Noise Added to Video	Average Mean Error (pixels) Across Dataset	Average Standard Deviation (pixels) Across Dataset
OpenCV	25 / 50 / 75	Populate from Table 1	Populate from Table 1
Blender	25 / 50 / 75	Populate from Table 1	Populate from Table 1
After Effects	25 / 50 / 75	Populate from Table 1	Populate from Table 1

Table 1. Average impact of noise on tracking performance for OpenCV, Blender, and After Effects.

Impact of Resolution on Tracking Performance

Tracker	Compared Video Resolution	Average Mean Error (pixels) Across Dataset	Average Standard Deviation (pixels) Across Dataset
OpenCV	480p / 720p / 1080p	Populate from Table 2	Populate from Table 2
Blender	480p / 720p / 1080p	Populate from Table 2	Populate from Table 2
After Effects	480p / 720p / 1080p	Populate from Table 2	Populate from Table 2

Table 2. Average impact of resolution changes on tracking performance across the dataset.

Qualitative Evaluation in Unreal Engine

To assess the perceptual impact of noise and resolution, camera paths from each system are imported into Unreal Engine. Virtual scenes reproduce key elements of the physical environment, enabling visual comparison between the original footage and rendered camera moves.

Example comparison of source and virtual scene

Figure 9. Example comparison between a source video (right) and a virtual scene (left) with matching elements in Unreal Engine, used to assess tracking performance qualitatively.

Discussion and Conclusion

The experiments show that all three tracking systems are sensitive to noise and resolution, but the degree and pattern of degradation differ. The OpenCV tracker generally achieves the lowest mean error per frame across noise levels, suggesting robustness to Gaussian noise. However, it is more affected by lower resolutions than Blender or After Effects, which maintain more consistent error trends as resolution decreases.

Qualitative inspection in Unreal Engine indicates that noise tends to produce jittery yet recognisable camera motion, while low resolution can cause more severe drift away from the original trajectory. At extreme combinations of noise and low resolution, tracks become unsuitable for precise virtual production work.

From a democratisation perspective, the open-source OpenCV tracker demonstrates that low-cost solutions can achieve performance comparable to, and sometimes surpassing, commercial tools, particularly when videos are recorded at sufficiently high resolution. Nevertheless, the complexity of configuration and the need to manage coordinate systems and data import pipelines in Unreal Engine remain barriers for non-expert users.

Future work could explore real-time tracking approaches, alternative open-source libraries, and improved tooling to simplify the integration of low-cost trackers into virtual production workflows, further expanding access to camera tracking technology.

References

Below is a minimal placeholder. For a full web reproduction, copy the reference list from the journal and format as numbered entries.

Include complete numbered references exactly as in the SMPTE article here.

BibTeX

@article{MunozLopez2026CameraTracking,
  author  = {Irene Mu{\~n}oz L{\'o}pez and Andrew Gilbert},
  title   = {Camera Tracking Systems and their Democratisation},
  journal = {SMPTE Motion Imaging Journal},
  year    = {2026},
  volume  = {26},
  number  = {1},
  month   = {January/February},
  doi     = {10.5594/JMI.2026/9GPF670}
}