Researchers from Nanjing University, JD.com, and six other Chinese universities introduced VideoJev, a family of models for video question answering. They answer about 100 times faster than models in Thinking mode and are almost as accurate. Picture an assistant that watches you cook through a camera. It keeps checking whether the chocolate is already in and which recipe step comes next. A video model in reasoning mode spends about 2.5 minutes on each answer. VideoJev, in contrast, works in two modes. It either picks one of the answer options that come with the question, or finds the right moment in the video from a description (temporal grounding). It answers in 1.1-1.5 seconds, and on average its accuracy is almost the same as that of reasoning models. Moreover, the model processes the video only once, no matter how many questions you ask about it.
In temporal grounding, VideoJev-4B and VideoJev-35B-A3B raised the mean mIoU (how well the found segment matches the correct one) by 6.7 and 10.8 points over their base models, Qwen3.5-4B and Qwen3.6-35B-A3B. At the same time, they run 2.8-3.1 times faster. The smaller model (65.3 mIoU) also beat the specialized TimeLens2-4B (62.0) and Gemini 2.5 Pro (60.4). The weights of both models are open under the Apache 2.0 license, along with the training, inference, and evaluation code. The datasets for temporal grounding are open too. However, the 240,000-question corpus for QA is not public. The models and datasets are on Hugging Face, the code is on GitHub, and more examples are on the project page.
How VideoJev handles video question answering
Regular video MLLMs write answers token by token. The authors dropped this generation step: they removed the next-token head from Qwen3.5 and Qwen3.6 and put an option scorer in its place. As a result, VideoJev cannot write a single word. It outputs only numbers, one score per option. On the other hand, the answer always comes from the given list, with a probability a program can use directly.
The model neither invents the options nor knows them in advance. Benchmarks take them from the dataset; in an application, the developer’s code or an LLM supplies them. You also cannot load the whole vocabulary into VideoJev and let it build the answer. Answers usually take several words, and building a phrase word by word is generation again. Besides, every option takes up context space. Up to 80% of the 256,000 tokens go to video frames, and the authors trained VideoJev on questions with 4-5 options.
So VideoJev can work in two ways:
- On its own. Finding a moment needs no ready-made options. Picking from a known list also needs no LLM if the code sets the list: recipe or instruction steps, action classes, scene states, or “yes” and “no”;
- Together with an LLM. VideoJev cannot answer an open question like “what is happening in this clip?” in its own words. So an LLM decides what to ask and proposes options, and VideoJev quickly checks them against the video.
The authors borrowed the idea from TypeSafe AI. Its model Jev also outputs probabilities for given options instead of text. TypeSafe calls such models System One, after Daniel Kahneman’s book Thinking, Fast and Slow. In the book, System One makes fast intuitive decisions, while System Two reasons slowly. Here, the LLM plays System Two, and VideoJev plays System One.
VideoJev architecture
The base is a pretrained video MLLM: the dense Qwen3.5-4B or the mixture-of-experts (MoE) Qwen3.6-35B-A3B, which uses about 3 of its 35 billion parameters per token. Along with the head, the authors remove the last layer’s MLP, because the features before it carry meaning better.
The input contains video tokens, an instruction, the question, and the candidates. A new <|option_end|> token follows each candidate, and a <|decide|> token closes the list. Linear projections map their hidden states to normalized 512-dimensional vectors: u for the decision and vi for each candidate. The candidate score is si = 10·u⊤vi + b, where b is a learned bias. This resembles the attention mechanism: the decision acts as the query, and the candidates as keys.
For temporal grounding, the candidates are the timestamps the model already sees next to frames. Each marks a two-frame segment, and the sigmoid of its score is the probability that the segment belongs to the event. In QA, the answer options are the candidates, and a softmax turns their scores into probabilities.
During training, a segment counts as positive if its midpoint lies inside the event. Binary cross-entropy (BCE) trains each score separately. Meanwhile, a soft IoU loss rewards covering the whole event and penalizes activations outside it. At inference, the model looks for the contiguous span with the largest sum of logits minus a threshold. This is the maximum subarray problem, which Kadane’s algorithm solves in linear time. In QA training, the authors shuffle the options, so the model cannot memorize where the correct answer sits. First, they train only the head on 30% of the videos. Then they fine-tune the language part too on the other 70%.
Video question answering without re-encoding
With causal attention, the video tokens come first and do not depend on the questions that follow. Therefore, the model only needs to run the video through itself once (prefill) and keep the result in the KV cache. There are two inference modes:
- Cached inference. Questions arrive one at a time, and each needs a single forward pass over its own text and candidates;
- Joint inference. All questions follow the video one after another, and one pass returns scores for all of them. Later questions see the text of earlier ones, but not their answers.
Joint inference is even slightly more accurate: it wins in 13 of 14 comparisons. The authors attribute this to training in the same format. For VideoJev-4B, going from one question to ten adds only 55 ms for temporal grounding and 31 ms for QA. It handles ten grounding queries in 0.39 seconds with an mIoU of 60.1. By comparison, Qwen3.5-4B needs 5.98 seconds without reasoning and 229 seconds with it, and reaches an mIoU of 52.3 and 47.8.
Results: temporal grounding and video question answering
The authors tested temporal grounding on Charades-STA, ActivityNet Captions, and QVHighlights. VideoJev-4B scored 65.3 mIoU on average at 0.276 seconds per query, and VideoJev-35B-A3B scored 67.7 at 0.336 seconds. Both versions lead on every metric and are the fastest open models of their size. The base models in these measurements also reuse the video prefix. So the speedup comes specifically from dropping timestamp generation.
For video question answering, the authors used LVBench, Video-MME, LongVideoBench, and MLVU without subtitles or audio. Compared with the base models without reasoning (Instruct), VideoJev gained 8.0 and 5.7 points. VideoJev-4B trails Qwen3.5-4B Thinking by 0.6 points, while VideoJev-35B-A3B beats Qwen3.6-35B-A3B Thinking by 0.7. Thinking mode is stronger on Video-MME and MLVU. However, VideoJev wins on LVBench, where videos last 68 minutes on average.
What matters under the hood
The authors also fine-tuned a regular Qwen3.5-4B to generate answers on the same data. It fell behind VideoJev-4B by 1.3 points of mIoU and 1.9 points in QA, so the gain does not come from the data alone. In addition, the soft IoU loss raised mIoU from 60.7 to 65.3. Finally, training both tasks together improved results on all seven benchmarks.
Model confidence
Candidate scores also work as confidence. On LVBench, the larger version is at least 99% sure of 591 answers. Of these, 95.8% are correct, versus 73.4% on average. So an agent can accept such answers right away and pass doubtful ones to a reasoning model. However, the model overestimates its confidence: the expected calibration error (ECE) is 8.65 percentage points for the smaller version and 8.13 for the larger one. Unlike Jev, VideoJev got no special training for calibration.
Limitations
Besides open-ended answers, VideoJev cannot do two more things: find several segments for one query and work with streaming video. Both versions also count small objects poorly. Asked how many books lay on a table, they answered “3” and “2” instead of “4”.
The authors measured latency only for forward passes on an H200, without video decoding. In practice, an answer will take longer. They see VideoJev not as an assistant but as a fast component for procedure monitoring, interactive helpers, robots, and agents that operate computers.
