ΑΙhub.org
Interview with Minji Park: humans in the loop to keep language models up to date
Minji Park is a PhD candidate at the Korea Institute of Energy Technology (KENTECH). In her paper, with co-authors Seunghyun Yoon and Hyuk Lim, she introduces a framework which aims to help make large language models stay more up-to-date when pieces of information change. We hear from Minji about inspired her work, how the framework operates, and her plans to further develop this research.
What is the topic of the research in your paper and why is it an interesting area for study?
The paper is fun, because we are challenging the norm. The usual AI models we use, such as ChatGPT, are like a black box. We don’t know what is going on inside the algorithm. We suggest that ‘humans also need to modify what is inside!’, and make the black more transparent.
When an assistant is used over months, some of what it was told stops being true. A meeting moves. A password is retired. A rule that held for one project was never meant for another. Recent work trains agents to keep the current value of a changed fact and to stop answering from stale ones, and it shows this is a real failure. Somebody has to decide that an old entry no longer counts, and our paper asks who that somebody is. Our answer is that that person should be the one who knows the meeting moved. So their correction should be an operation on the state the assistant actually reads, not one more sentence added to a context window that a later turn may reassemble without it.
Many years ago, I learned 3D printing and arts. The idea evolved from there, looking at the computer as art. Transforming information into colors and hues. At this point, you might be saying, ‘huh?’, but please continue reading and this will make sense!
Could you tell us about the framework that you’ve introduced?
Transfiver (that stands for Transparent Framework for Interactive, Verifiable, Editable Representation. It is not a secret that we wanted to advance transformers) rests on one requirement, which we call the one-state contract. What it means is that we humans need to see what the machine is seeing, and it should be one live state. Editing what you see edits the computation. Restoring it restores the computation.
Suppose the AI assistant holds an entry you have retracted. You can leave this entry out in the context of the current turn, which is what retrieval systems do. Or you can retract it in the state. On that turn the two are indistinguishable and the model sees exactly the same lines. However, information hidden for one time only may reappear in the next conversation. That is why the contract has to be about the state.
I was inspired to look into this because I was using LLMs and they were making up words and it confused me. There were so many-dashed-words and words that are not normally used in a certain context were everywhere. So I had to ask the model to “talk simple”. Immediately after this instruction the model pardons itself and explains in a kind manner. However, after a few more turns, it forgets. The sentences that it generates are too complicated. After months and months of repeating myself “be comprehensible” to the model, I asked, ‘what if we edited it at a more fundamental level before it makes a mistake?’
In our paper, the state changes along two routes. In a stream update (which is updated by the model), the model reads the ongoing conversation and decides whether new information revises an existing entry or opens a new one. In a directed edit (which is updated by humans) a person opens an addressed entry and changes it. Both routes write to the same state(!).
A conceptual image of Transfiver. Note: this does not portray the actual framework.
What are the main contributions of your research?
I have this ultimate grand dream about changing everything: the model’s workings, the interface, and the way we communicate.
However, in our research, a small step towards this, we introduce a simple framework where humans have to make an edit. In the paper, first, we specify the distinctions the state has to keep apart. Then, we turn “memory, reasoning and output in one living state” from a slogan into contract types, which is what separates this from a memory add-on. A small implementation follows. In this way, personalizing the AI can be an advantage. Also, we humans will be less annoyed from seeing the models failing to help the user. This is just a beginning though. Please stay tuned for further research.
How did you go about testing your framework and how did it perform?
Turns out… better than I expected. Current LLMs demonstrate hallucination, and they do not generate 100% accurate information even though they have the access to all the factual information. With state retraction, the performance on the next turn increased! (I mean, of course it should improve.)
For serious readers, here are some numbers. The first test uses 44 items from a public benchmark of multi-session conversations, where a user states a value and later replaces it. We compared two ways of dealing with the models. Deleting the line from the prompt and retracting it in the state give the same accuracy when you intervene, 0.659 for Mistral-7B, 0.705 for Qwen2.5-7B, and 0.659 for Phi-3-mini. Then the next turn arrives. The deleted line comes back in 31 of 44 items. The retracted line comes back in none of them.
Interested? Please visit our paper on arXiv!
What further improvements are you planning?
The reported testing was done on a benchmark, and not tested interactively with a live user. In real life, humans also make mistakes and sometimes auto correction works. This is the gap between labs and the real world.
However, with small steps that I walk everyday, ultimately, I want to make the system vibrant, literally. Make it visible, give colors and layers. Make a small AI galaxy that acts as an assistant LLM, so that humans can intuitively comprehend the AI.
(The details are yet to be revealed, but I am very keen to share it with the world. Stay tuned to arXiv!)
Our goal is towards a personalized partner.
About Minji
|
Minji Park is a PhD candidate in Energy AI at the Korea Institute of Energy Technology (KENTECH) in South Korea, where she works with Professor Hyuk Lim. Her research is on fundamental architectures for language models. Her wider interests include transparency, safety, and interpretability. Her work will appear at theprimary.com soon! She has worked as a red-teaming reviewer and annotator for AI systems, and was previously an English–Korean interpreter. Both jobs involved judging output against a contextual interpretation, beyond what is seen or heard. |
AUAI is supported by:
