Detection Algorithms

scenedetect.detectors Module

This module contains implementations of scene detection algorithms by inhereting from the base SceneDetector class (in scenedetect.scene_detector) and implementing the required methods. This allows implementation of other generic algorithms as well as custom scenario-specific algorithms.

The following detection algorithms come with PySceneDetect:

  • ContentDetector:

    Detects shot changes by considering pixel changes in the HSV colorspace.

  • ThresholdDetector:

    Detects transitions below a set pixel intensity (cuts or fades to black).

  • AdaptiveDetector:

    Two-pass version of ContentDetector that handles fast camera movement better in some cases.

ContentDetector

ContentDetector compares the difference in content between adjacent frames against a set threshold/score, which if exceeded, triggers a scene cut.

This detector is available from the command-line as the detect-content command.

class scenedetect.detectors.content_detector.ContentDetector(threshold=27.0, min_scene_len=15, luma_only=False)

Detects fast cuts using changes in colour and intensity between frames.

Since the difference between frames is used, unlike the ThresholdDetector, only fast cuts are detected with this method. To detect slow fades between content scenes still using HSV information, use the DissolveDetector.

Parameters
  • threshold (float) – Threshold the average change in pixel intensity must exceed to trigger a cut.

  • min_scene_len (int) – Once a cut is detected, this many frames must pass before a new one can be added to the scene list.

  • luma_only (bool) – If True, only considers changes in the luminance channel of the video. The default is False, which considers changes in hue, saturation, and luma.

DELTA_H_KEY = 'delta_hue'
DELTA_S_KEY = 'delta_sat'
DELTA_V_KEY = 'delta_lum'
FRAME_SCORE_KEY = 'content_val'
METRIC_KEYS = ['content_val', 'delta_hue', 'delta_sat', 'delta_lum']
get_metrics()

Get Metrics: Get a list of all metric names/keys used by the detector.

Returns

List of strings of frame metric key names that will be used by the detector when a StatsManager is passed to process_frame.

is_processing_required(frame_num)

[DEPRECATED] DO NOT USE

Test if all calculations for a given frame are already done.

Returns

False if the SceneDetector has assigned _metric_keys, and the stats_manager property is set to a valid StatsManager object containing the required frame metrics/calculations for the given frame - thus, not needing the frame to perform scene detection.

True otherwise (i.e. the frame_img passed to process_frame is required to be passed to process_frame for the given frame_num).

process_frame(frame_num, frame_img)

Similar to ThresholdDetector, but using the HSV colour space DIFFERENCE instead of single-frame RGB/grayscale intensity (thus cannot detect slow fades with this method).

Parameters
  • frame_num (int) – Frame number of frame that is being passed.

  • frame_img (ndarray) – Decoded frame image (numpy.ndarray) to perform scene detection on. Can be None only if the self.is_processing_required() method (inhereted from the base SceneDetector class) returns True.

Returns

List of frames where scene cuts have been detected. There may be 0 or more frames in the list, and not necessarily the same as frame_num.

Return type

List[int]

scenedetect.detectors.content_detector.calculate_frame_score(current_frame_hsv, last_frame_hsv)

Calculates score between two adjacent frames in the HSV colourspace. Frames should be split, e.g. cv2.split(cv2.cvtColor(frame_data, cv2.COLOR_BGR2HSV)).

Parameters
  • curr_frame_hsv – Current frame.

  • last_frame_hsv (Iterable[ndarray]) – Previous frame.

  • current_frame_hsv (Iterable[ndarray]) –

Returns

Tuple containing the average pixel change for each component as well as the average across all components, e.g. (avg_h, avg_s, avg_v, avg_all).

Return type

Tuple[float]

AdaptiveDetector

AdaptiveDetector compares the difference in content between adjacent frames similar to ContentDetector except the threshold isn’t fixed, but is a rolling average of adjacent frame changes. This can help mitigate false detections in situations such as fast camera motions.

This detector is available from the command-line as the detect-adaptive command.

class scenedetect.detectors.adaptive_detector.AdaptiveDetector(adaptive_threshold=3.0, luma_only=False, min_scene_len=15, min_delta_hsv=15.0, window_width=2, video_manager=None)

Detects cuts using HSV changes similar to ContentDetector, but with a rolling average that can help mitigate false detections in situations such as camera moves.

Parameters
  • adaptive_threshold – Threshold value (float) that the calculated frame score must exceed to trigger a new scene (see frame metric adaptive_ratio in stats file).

  • luma_only – Only consider luma/brightness channel (useful for greyscale videos).

  • min_scene_len – Minimum length of any scene.

  • min_delta_hsv – Minimum threshold (float) that the content_val must exceed in order to register as a new’ scene. This is calculated the same way that detect-content calculates frame score.

  • window_width – Size of window (number of frames) before and after each frame to average together in’ order to detect deviations from the mean.

  • video_manager – [DEPRECATED] DO NOT USE. For backwards compatibility only.

ADAPTIVE_RATIO_KEY_TEMPLATE = 'adaptive_ratio{luma_only} (w={window_width})'
get_content_val(frame_num)

Returns the average content change for a frame.

get_metrics()

Combines base ContentDetector metric keys with the AdaptiveDetector one.

post_process(_)

After an initial run through the video to detect content change between each frame, we try to identify fast cuts as short peaks in the content_val value. If a single frame has a high content-val while the frames around it are low, we can be sure it’s fast cut. If several frames in a row have high content-val, it probably isn’t a cut – it could be fast camera movement or a change in lighting that lasts for more than a single frame.

process_frame(frame_num, frame_img)

Similar to ThresholdDetector, but using the HSV colour space DIFFERENCE instead of single-frame RGB/grayscale intensity (thus cannot detect slow fades with this method).

Parameters
  • frame_num (int) – Frame number of frame that is being passed.

  • frame_img (Optional[int]) – Decoded frame image (numpy.ndarray) to perform scene detection on. Can be None only if the self.is_processing_required() method (inhereted from the base SceneDetector class) returns True.

Returns

Empty list

stats_manager_required()

Overload to indicate that this detector requires a StatsManager.

Returns

True as AdaptiveDetector requires stats.

ThresholdDetector

ThresholdDetector uses a set intensity as a threshold to detect cuts, which are triggered when the average pixel intensity exceeds or falls below this threshold.

This detector is available from the command-line as the detect-threshold command.

class scenedetect.detectors.threshold_detector.ThresholdDetector(threshold=12, min_scene_len=15, fade_bias=0.0, add_final_scene=False, block_size=None)

Detects fast cuts/slow fades in from and out to a given threshold level.

Detects both fast cuts and slow fades so long as an appropriate threshold is chosen (especially taking into account the minimum grey/black level).

Parameters
  • threshold (float) – 8-bit intensity value that each pixel value (R, G, and B) must be <= to in order to trigger a fade in/out.

  • min_scene_len (int) – FrameTimecode object or integer greater than 0 of the minimum length, in frames, of a scene (or subsequent scene cut).

  • fade_bias (float) – Float between -1.0 and +1.0 representing the percentage of timecode skew for the start of a scene (-1.0 causing a cut at the fade-to-black, 0.0 in the middle, and +1.0 causing the cut to be right at the position where the threshold is passed).

  • add_final_scene (bool) – Boolean indicating if the video ends on a fade-out to generate an additional scene at this timecode.

  • block_size – [DEPRECATED] DO NOT USE. For backwards compatibility.

THRESHOLD_VALUE_KEY = 'delta_rgb'
get_metrics()

Get Metrics: Get a list of all metric names/keys used by the detector.

Returns

List of strings of frame metric key names that will be used by the detector when a StatsManager is passed to process_frame.

Return type

List[str]

post_process(frame_num)

Writes a final scene cut if the last detected fade was a fade-out.

Only writes the scene cut if add_final_scene is true, and the last fade that was detected was a fade-out. There is no bias applied to this cut (since there is no corresponding fade-in) so it will be located at the exact frame where the fade-out crossed the detection threshold.

Parameters

frame_num (int) –

process_frame(frame_num, frame_img)
Parameters
  • frame_num (int) – Frame number of frame that is being passed.

  • frame_img (numpy.ndarray or None) – Decoded frame image (numpy.ndarray) to perform scene detection with. Can be None only if the self.is_processing_required() method (inhereted from the base SceneDetector class) returns True.

Returns

List of frames where scene cuts have been detected. There may be 0 or more frames in the list, and not necessarily the same as frame_num.

Return type

List[int]

scenedetect.detectors.threshold_detector.compute_frame_average(frame)

Computes the average pixel value/intensity for all pixels in a frame.

The value is computed by adding up the 8-bit R, G, and B values for each pixel, and dividing by the number of pixels multiplied by 3.

Parameters

frame (ndarray) – Frame representing the RGB pixels to average.

Returns

Average pixel intensity across all 3 channels of frame

Return type

float