Our Research.

Multimodal models trained on next-token prediction or contrastive objectives develop strong single-frame and short-context representations. They do not learn temporal event structure. They do not model how semantic states transition across time, what causal dependencies connect events, or how intent propagates through a sequence. These properties do not emerge reliably from scale alone — they require supervision signals and representation targets explicitly designed around temporal structure.

TEMPO researches learned temporal event representations for multimodal sequences. The central hypothesis is that reliable long-horizon multimodal reasoning requires explicit modeling of event boundaries, causal dependencies, and semantic state transitions as first-class representational objects — not as implicit correlates of frame-level features. We develop and evaluate these representations using STR artifacts as supervision targets, which provide semantically dense, temporally structured training signal at significantly lower cost than pixel-space or latent-space generation objectives.

Training on real human decisions in production — rather than synthetic benchmarks or web-scraped video — is a deliberate methodological choice. Human editorial decisions encode implicit temporal reasoning: what matters, when it matters, and why it relates to what preceded it. This signal is not available in standard video datasets and is not reproducible outside a production deployment surface.

The program runs in three stages, each producing research artifacts integrated into production before the next begins.

Stage 1: Temporal event representation architecture. Long-context evaluation harness designed for causal coherence and semantic consistency under distribution shift. Production integration for real-world measurement against live human signal.

Stage 2: Cross-domain generalization of temporal representations. Robustness under input perturbation and context length scaling. Evaluation against external multimodal baselines through the UMI.

Stage 3: Long-horizon sequential reasoning across extended multimodal contexts. Cross-task transfer of temporal representations beyond the training domain. Selective external disclosure aligned with IP and partnership strategy.

Heterogeneous media inputs — video sequences, audio waveforms, images — are normalized into modality-specific token sequences prior to model inference. The MFI supports both early fusion, where cross-modal token sequences are jointly encoded into a shared representation space, and late fusion, where per-modality encodings are projected into structured data for models that process modalities independently. This abstraction decouples upstream media encoding from downstream model architecture, enabling consistent input representations across providers with different tokenization strategies and context window constraints.

Model outputs are constrained to a validated, parametrized schema encoding semantic and temporal structure rather than pixel-space or latent-space generation targets. STR artifacts are typed, versioned, and explicitly represent event boundaries, causal relations, and semantic state transitions as discrete, inspectable objects.

This formulation decouples video understanding from video generation during training. STR targets preserve the informational density required for temporal reasoning supervision while avoiding the computational overhead of autoregressive or diffusion-based generation. The artifacts are directly comparable across model variants, providing a tractable and consistent supervision signal for training temporal event models. This decoupling is productive both for research — enabling rigorous comparative evaluation across architectures — and for scaling, where it reduces training compute requirements by orders of magnitude relative to generation-based supervision targets.

Video understanding and video generation can be decoupled during research and development and recoupled selectively.

Structured Temporal Representations are executed through a GPU-accelerated rendering pipeline that serves as the terminal stage of the model inference loop. The rendering engine accepts STR artifacts as input and produces deterministic media outputs, with execution behavior that is fully specified by the artifact and independent of the model that produced it.

Because AI-generated and human-authored STR artifacts share the same schema and pass through the same execution environment, outputs are produced under identical computational conditions. This makes the rendering engine a native evaluation substrate: differences in output quality are attributable to the upstream representation, not to execution variability. This property is load-bearing for benchmarking TEMPO models against both external baselines and human decisions within the same pipeline.

The UMI defines the input-output contract for all model interactions within the Rkive stack. On the input side it enforces MFI-normalized representations, ensuring architecture-agnostic media encoding across providers. On the output side it constrains model responses to validated STR schemas, ensuring that outputs from different models — external providers and internal research models including TEMPO — are structurally comparable.

This bidirectional standardization enables native cross-model evaluation without additional instrumentation. Model selection, fallback routing, and comparative benchmarking operate at the interface level, transparent to both the product layer above and the execution layer below. The UMI is what makes the system model-agnostic in a technically precise sense: no component above or below it carries provider-specific assumptions.