MSP Library (Motion Signal Processor)

MSP (Motion Signal Processor) is a cross-platform motion analysis engine designed to evaluate human movement data against pre-trained MoveSpace Models (.msm).

This library is a clean-room implementation of the scoring logic used in the Just Dance game engine. It parses proprietary binary classifier files and processes real-time accelerometer data to calculate precision scores, energy intensity, and directionality.

Core Concepts

What is a .msm file?

MSM (MoveSpace Model) is a proprietary binary format used by the game engine.

• Single Move: Unlike generic ML models, an .msm file represents a single, specific gesture (e.g., "generic_generic.msm").
• Binary Classifier: It contains pre-calculated statistical means, covariance matrices, and specific thresholds (Energy, Shake, Direction) tailored for that specific timeframe of the choreography.
• Dynamic Loading: In a game context, these files are loaded and unloaded dynamically as the song progresses based on the timeline.

Features

• Binary Parsing: Native support for parsing .msm binary blobs (Supports Format Versions 5 through 8).
• Signal Processing: Real-time 60Hz signal smoothing, derivative calculation, and 3D vector normalization.
• Scoring Algorithms:
  • Mahalanobis Distance: Used for complex moves with high variance.
  • Naive Bayes: Used for simpler, more static moves.
• Advanced Metrics:
  • Energy Factor: Determines if the player put enough force into the move.
  • Direction Tendency: Detects if a move was performed in the inverse direction (e.g., Left vs Right).
  • Shake Detection: Identification of rapid back-and-forth signals.

Language Support & Status

While the codebase shares a unified logic structure across languages, the maturity of the implementations varies.

Language	Status	Notes
JavaScript	✅ Stable	Original reference implementation. Used in production.
TypeScript	✅ Stable	Supported via JS module/definitions.
C++	⚠️ Experimental	Ported, compiles, but untested.
Python	⚠️ Experimental	Ported, logic verified, but untested in live scenarios.

Usage Guide (JavaScript)

The library exposes a ScoreManager that handles the lifecycle of analyzing a single move.

1. Installation

import MSP_LIB from './scoring/msp.js';

2. Initialization

Initialize the engine once.

const msp = new MSP_LIB.ScoreManager();
const gameInterface = msp.Game();

// Init(StatDistLow, StatDistHigh, AutoCorr, DirImpact, SmoothingFreq)
// Passing -1 forces the engine to read the optimal thresholds from the .msm file itself.
// 60.0 is the standard sensor polling rate (Hz).
gameInterface.Init(-1, -1, -1, -1, 60.0);

3. Start Analysis

When a move starts in your timeline, load the specific .msm binary.

// buffer: ArrayBuffer containing the binary data of 'SpecificMove.msm'
// duration: The theoretical duration of the move in seconds (e.g., 4.0)
const success = gameInterface.bStartMoveAnalysis(buffer, duration);

if (!success) {
    console.error("Failed to parse MoveSpace Model.");
}

4. Feed Sensor Data

During the move, feed normalized accelerometer data.

• Progress: 0.0 (Start) to 1.0 (End).
• Accels: Normalized to G-Force (1.0 = 9.81 m/s²).

// Call this in your game loop
function updateScoring(currentProgress, accelX, accelY, accelZ) {
    gameInterface.UpdateFromProgressRatioAndAccels(
        currentProgress, 
        accelX, 
        accelY, 
        accelZ
    );
}

5. Finish & Score

When progress >= 1.0, finalize the calculation.

gameInterface.StopMoveAnalysis();

// 1. Accuracy Score (0.0 to 1.0)
const accuracy = gameInterface.fGetLastMoveRatioScore();

// 2. Energy Factor (Float, usually around 1.0)
// > 1.0 = High Energy, < 1.0 = Low Energy
const energy = gameInterface.fGetLastMoveEnergyFactor();

// 3. Direction (Tools Interface)
// Positive = Correct Direction, Negative = Wrong Direction
const directionCheck = msp.Tools().fGetDirectionTendencyImpactOnScoreRatio();

console.log(`Score: ${(accuracy * 100).toFixed(2)}%`);

API Reference

ScoreManager

The main controller class.

GameInterface

• Init(...): Sets up the smoother and default thresholds.
• bStartMoveAnalysis(buffer, duration): Parses the .msm binary.
• UpdateFromProgressRatioAndAccels(...): Feeds data points.
• StopMoveAnalysis(): Finalizes the statistical math.
• fGetLastMoveRatioScore(): Returns the final accuracy percentage (0-1).

ToolsInterface

• fGetLastMoveStatisticalDistance(): Returns the raw mathematical distance (lower is better).
• ulGetMoveCustomizationFlagsFromFileData(...): Reads binary flags (e.g., does this move ignore direction?).
• pstGetMoveClassifierStruct(): Returns the parsed internal structure (means/covariances) for debugging.

License

This project is part of the LilyPad Next Licensed under the Apache License 2.0