Jev AI's no-hallucination claim draws scrutiny after RLHF creator's admission

Diogo Almeida, who proclaims himself a ChatGPT “co-creator”, is running a viral marketing push for a new AI called Jev. The model is built on his work with RLHF, the OpenAI chatbot's core technology.

However, he is now acknowledging that his own creation has shortcomings.

Almeida's startup TypeSafe came out of stealth this month, securing $40 million in seed funding. The company's pitch claimed he had solved the hallucination problem he encountered while working on ChatGPT.

But the pitch for the Jev model itself is less compelling and prompts some doubts.

To begin with, his assertion that he “co-created ChatGPT” is not entirely accurate.

Almeida appears fourth among 20 authors on the 2022 InstructGPT paper, which used RLHF on language models. That paper was just one of several resources ChatGPT developers drew on.

His role is further diminished by a footnote in the PDF, which lists Almeida as one of nine “primary” authors, not a lead.

Furthermore, RLHF itself existed almost five years before that paper, and its foundational method was developed by six different researchers.

Jev: an AI model built on a career shortcoming

Almeida is trying to launch Jev and attract funding by conceding that RLHF, his own invention, “is not useful.”

He describes ChatGPT as “unreliable without human supervision” and attributes to RLHF issues like “mode dropping, overconfidence, and an overall lack of reliability.”

Almeida also uses the opportunity to assert that Jev “can't hallucinate.”

However, his assertion about anti-hallucination tech quickly unravels upon scrutiny.

According to Jev's own docs, its defense against hallucinations comes from a design that picks options instead of generating sentences, with a strict limit of 255 choices — not from any remarkable code or algorithmic leap.

Jev essentially acts as a basic decision selector, choosing from a restricted set of options or preset outputs, like estimating a probability percentage.

Since Jev does not produce text via a conventional chatbot, it cannot hallucinate — hallucinations involve generating phrases that form sentences or factual assertions.

Jev can only select from predetermined outputs and does not generate interpretable sentences. Hence, Almeida boasts that hallucinations are inherently impossible in Jev.

Jev is only days old and remains in early access. Input tokens cost $42 per billion, while output tokens are currently free as part of subsidies. Almeida says Jev offers advantages in speed, cost, and quality over rivals.