Then the llm is constrained to a few special tokens indicating the possibilities? e.g. <option1> <option2>
rene_d 4 hours ago [-]
To get calibrated probabilities sounds like a very good feature, if they are indeed well calibrated.
And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
But I still need to experiment with either Jev/SemIf myself.
vova_hn2 2 hours ago [-]
> To get calibrated probabilities sounds like a very good feature, if they are indeed well calibrated.
This is a valid point.
> And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
Literally every inference framework supports constrained encoding. You can make the model choose only from allowed tokens and you can infer only the first diverging token.
It's baffling to me that no inference provider actually exposes this functionality, so you have to run the model yourself to do it.
allanmacgregor 2 hours ago [-]
I thought this wasn't really an LLM sure text as input but the folks at Typesafe have been very adamant this is a new class of model not just another LLM.
Although to be fair we don't know enough about the architecture.
hresvelgr 4 hours ago [-]
I think "Black boxes are back in fashion" is missing the point. I think LLMs are still largely black boxes, and I don't think chain of thought is representative of any degree of inner machination. Asking it questions to justify itself is at best a facsimile, and for the most part it's useful, but it's fundamentally a facsimile.
Where I understand Jev to be a significant jump is that afaik the confidence scoring is actually derived from the normalised probabilities, and not a continuation in a chain of prediction masquerading as "confidence."
aszen 5 hours ago [-]
About jev being a black box and the potential for bias, I think it boils down to what questions you are asking the model.
Broad questions like Is this resume good / score this city will ofcourse be biased but I think jev encourages more granular focused questions like Score this candidates Python experience / Rate this city for its food which then allows you to introduce your own biases in which questions you ask and how you combine their answers.
In this way I think jev like models can be easier to reason about for critical decisions.
Mumps 2 hours ago [-]
This is off topic.
Do people not feel like LLM speak (Claudisms) is infecting their own diction? Saying 'a new "shape" of LLM' sits so very poorly.
And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
But I still need to experiment with either Jev/SemIf myself.
This is a valid point.
> And in my experiments even Qwen 3.8 has a hard time to consistenly conform to a schema, requiring retries, JSON cleanup etc, so to have a model of similar quality (SemIf et al) that simply cannot deviate from the schema by construction could be very helpful.
Literally every inference framework supports constrained encoding. You can make the model choose only from allowed tokens and you can infer only the first diverging token.
It's baffling to me that no inference provider actually exposes this functionality, so you have to run the model yourself to do it.
Although to be fair we don't know enough about the architecture.
Where I understand Jev to be a significant jump is that afaik the confidence scoring is actually derived from the normalised probabilities, and not a continuation in a chain of prediction masquerading as "confidence."
Broad questions like Is this resume good / score this city will ofcourse be biased but I think jev encourages more granular focused questions like Score this candidates Python experience / Rate this city for its food which then allows you to introduce your own biases in which questions you ask and how you combine their answers.
In this way I think jev like models can be easier to reason about for critical decisions.
Do people not feel like LLM speak (Claudisms) is infecting their own diction? Saying 'a new "shape" of LLM' sits so very poorly.