// HACKER NEWS — CYBERSECURITY
A single function Jev-like wrapper for LLMs, including vision models
I was intrigued by Jev and the self-hostable projects appearing around it, such as OpenJev and SemIf. Reading about them introduced me to a neat trick: reading an LLM's token probabilities.
Apparently this is an old trick for some people. See e.g. OpenAI's logprobs cookbook. But it was new to me.
I believe the basic idea is to write a prompt like this:
Then add a few JSON request parameters to a compatible Chat Completions request:
The LLM API will return the letter plus the model's log probabilities for alternative tokens.
Repeat for each question. Forcing it to generating only one token avoids a lengthy answer and is super quick, though processing the input still costs time. Though for each of the questions a shared state prefix can be KV-cached if the backend supports it.
The fun part: this works with vision models too. Jev's documented request format currently describes only text/JSON state. I added an attachments field for images for my local experiments.
My example captures webcam frames, sends base64 JPEGs, and prints a table: is a person visible, are we indoors or outdoors, and how bright is the scene? With Gemma 4 12B on my RTX 3090, I get around 1 frames per second, with three questions per frame. I also ran it against OpenAI gpt-6-luna and got around 0.2 FPS. Presumably because I didn't make any effort to avoid the cost of a separate connection through their system per question per frame.
Specialized computer vision models surely are much more efficient, but what I like here is the flexibility: change a condition by describing it in plain text.
Here's the standalone Python example (OpenCV is just used for convenient access to the webcam, not for any actual computer vision):