Bilal Shihab / Projects

Civassist

A desktop app my team is building that tells people in Texas whether they may qualify for Medicaid, CHIP, WIC and SNAP, using a local model, RAG over the state's program documents and a rulebook of cited eligibility rules, with everything kept on their own computer:

Collaborated with Jay Damodaran and Kaylan TchamdjouJun 2026 – presentIn development
Spec
Model
Gemma 3 4B, Q4_K_M, llama-server
Backend
FastAPI, one local process
Programs
Medicaid, CHIP, WIC, SNAP (Texas)
Screening questions
14
Eligibility rules
46, each cited to its source
Checker tests
45, all passing
Retrieval
LlamaIndex, ChromaDB, bge-small-en-v1.5
Corpus
1,339 passages of 256 tokens
The Civassist chat page: a mother says she has a 4-month-old and earns $1,800 a month, and Civassist asks follow-up questions
Fig. 1A test conversation I ran for this page with a made-up mother and her 4-month-old daughter. After each message, a second model call fills in the screening form and the rules engine runs again. The run also shows what is left to fix: the form never recorded the baby's age, so the model kept asking for it, and it logged her savings as over $5,000 when she had said under $1,000, so the engine correctly turned down SNAP on a wrong answer.
The disclaimer that opens the app: provided as is, data processed locally, and an I Accept and Agree button
Fig. 2The disclaimer Kaylan built, shown before the first message.
The Documents page, with no uploaded files and three official documents: wic_eligibility.txt, texas_works_income_limits.txt and chip_eligibility.txt
Fig. 3The files page Jay built, with the retrieval pipeline's three source documents: the Texas Works Handbook (the 1.2 MB file) and the state's CHIP and WIC pages. Files a person uploads will go through the document parsers once those are connected.

Why I built it

I wanted to work on something that helps people directly. To find out whether your family qualifies for Medicaid, CHIP, WIC or SNAP in Texas, you have to work through income charts and a state handbook that runs to hundreds of pages, and the answer depends on details like household size, age and pregnancy. Civassist asks those questions in a normal conversation and gives an answer. It runs entirely on the person's own computer, so their income, immigration status and family details never leave it.

The team split the work three ways. Jay built the chat interface, the files page, the document parsers and the model server, including the two-call flow where one model call talks with the person and a second pulls their answers into a fixed form. Kaylan built the disclaimer, the heartbeat that shuts the local server down when the page closes, and the test setup: a stand-in model, browser tests of the interface and automatic test runs on GitHub. I built the retrieval pipeline and the eligibility engine.

The hard part

The hard part was getting the rules right, and showing they were right. Income limits and other hard facts live in a rulebook that plain Python reads, never in the model, so the model can't invent a limit. That only helps if the rulebook matches the handbook. The rules were transcribed from the handbook with help from Claude, so every rule carries the section it came from, and most carry the exact quote.

To check them, I built an eval in which a judge model reads each rule next to the source text it cites and flags anything the source doesn't support. To test the judge itself, I planted five false rules, from blatant to a boundary flipped by one cent, and it caught all five. Three judges (the local Gemma, Gemini 3.6 Flash and Claude Sonnet) found no transcription errors. Each flag traced to how the harness fed them evidence or to a table whose columns were scrambled in the copy-pasted handbook chart, which I confirmed by arithmetic.

A second eval looks for the opposite problem, a rule the handbook states that the rulebook leaves out. I deleted a real rule and checked that the judge named it, then ran it on the full rulebook. It found three missing rules, including newborns of mothers on Medicaid, who qualify with no income test, and I added them. A last sweep extracted every requirement from the whole source corpus. It found no missing rules but two places where my checker didn't apply a rule the rulebook already stated, and both wrongly turned people down. Both are fixed, with tests.

The code is in a private team repository.