Skip to main content
DoneThat

AI Adoption GuideEducationTeach

Intelligent Tutoring System

A RAG-based tutor answers student questions from course materials and adapts explanations based on prior student responses.

Education processRecruitAdmitEnrollTeachAssessCredentialGraduateAdvance

By Don, DoneThat’s AI coach · updated

Ground the tutor in this course, not the open web

An intelligent tutoring system for a credit-bearing course answers student questions from the assigned corpus and withholds an answer when retrieval returns nothing. The quality bar is binary: every substantive reply names the lecture, reading, slide deck, or lab handout it used, or the student sees a refusal with no invented explanation. The model must not fill gaps with a lecture that was never given, a chapter that was never assigned, or a policy exception faculty never wrote.

A general-purpose chatbot can sound helpful while teaching next year's textbook edition, a competing instructor's public slides, or a grading curve that does not exist in this syllabus. Students treat fluent prose as authority. Faculty still own the syllabus and the grade. The tutor is a study aid for materials already in the learning environment. It is not a second instructor and it does not replace office hours.

LMS suites such as Canvas, Blackboard, Moodle, and Anthology already hold the files and dates students are supposed to use. Treat those products as the class of places the corpus lives, not as a ranked feature list. The tutoring layer should read what faculty published there, or in a mirrored course pack, and nothing else when the question is about course content.

A knowledge graph from syllabus can tell the retriever which week, module, and learning outcome a file belongs to. That graph is an index, not extra teaching. If the graph and the files disagree, the files win.

Retrieve first, then answer with citations

Restrict the retriever to this course's pack: lecture notes, assigned PDFs, captioned slide decks, lab manuals, and the published syllabus. Exclude the open web, other sections, and unpublished drafts unless faculty added them.

On each question, retrieve passages before any explanation is generated. Do not draft until retrieval returns ranked snippets with stable identifiers (filename, page or slide, module or week). Write the answer only from those snippets. Quote or tightly paraphrase, and attach citations the student can open in the same LMS or course pack.

If retrieval is empty, or the top hits fall below a faculty-set relevance floor, return no content answer. Tell the student the question sits outside the posted materials and point them to office hours or the syllabus contact line, without inventing a reading.

Log every turn for faculty review: timestamp, the student identity the LMS already uses, the question, retrieved document IDs, the answer or the refusal, and a flag if the student asked for a grade, extension, or policy exception. Citations are how both sides check the tutor. Faculty who cannot see which file was retrieved cannot catch a wrong edition or an answer key in the wrong folder.

One worked example: a student in an undergraduate research-methods course asks whether they can skip the IRB tutorial because they will only analyze public social-media posts. Retrieval finds the week-three lecture on human-subjects review and the syllabus paragraph that requires the tutorial for every project, including secondary analysis of public posts. The tutor quotes those two sources, cites them, and stops. It does not add that many universities waive the requirement. If the same student then asks what the department chair told last year's cohort about waivers, retrieval is empty. The tutor refuses and writes the turn to the faculty log.

Automated content generation can produce practice items from the same corpus. Keep generation and tutoring separate. Generated drills must not enter the tutor's retrieval set until faculty approve them as course materials.

Stay silent when the corpus has nothing

Empty retrieval is a success state. The failure mode is answering from general web knowledge: the model already "knows" how ANOVA works or how a typical syllabus handles late work, and it will speak as if that knowledge were this course. Students cannot tell the difference. Faculty often discover it after a midterm, when a procedure from a generic textbook appears in student work and not in the lab manual.

Gate generation on retrieval. No snippets, no answer body. A one-line refusal plus a pointer to office hours is enough. Do not generate a mini-lecture "just this once," summarize what most textbooks say, or fetch a new source at runtime.

If Friday's lecture is not in the corpus yet, the tutor must not reconstruct it from last year's unposted recording, a colleague's public deck, or the model's training data. Inventing a lecture that is not in the corpus fails the quality outcome even when the invented lecture is pedagogically sound.

When students hit refusals repeatedly on a topic faculty thought was covered, treat that as a corpus problem. The file may be missing, chunking may have skipped the relevant slides, or an adaptive learning path engine may be sending students to a module whose materials were never uploaded. Fix the pack. Do not loosen the gate.

Keep chat out of the gradebook

Do not treat tutor chat as graded work. A conversation that helped a student rehearse a derivation is not a submitted homework file, not an exam response, and not attendance. If the LMS gradebook can ingest transcripts, leave that mapping off. Faculty still assign and score the work they designed.

Say this in the student-facing chrome and in the syllabus language beside the tool: the tutor is optional practice against posted materials; its answers are not official; using it does not complete an assignment. Comments on writing the student will actually submit belong in formative feedback generation on the draft itself, with the same citation rules, not in a scored tutor thread.

Do not invent a policy exception. Questions about deadlines, extra credit, incomplete grades, attendance, disability accommodations, and whether the tutor is allowed on an exam are policy, not content. Even when retrieval finds a syllabus paragraph, quote it with a citation and send the student to faculty or the published process for anything that sounds like a one-off exception. The model must not say the student should be fine, or that it will mark them excused. Those sentences become screenshots.

Office hours remain the place for judgment calls, stuck work after the assigned examples, and anything the corpus cannot settle. The tutor should cut repeat questions about a page that is already posted. It does not replace the human who can see that a student is lost, unwell, or asking for a waiver.

Review logs before you teach the topic again

Faculty review is part of the operating loop. Sample the logs for missed retrieval, citations that pointed at the wrong week, grade-change requests in chat, and refusals that cluster on a topic you thought was clear.

Use the log to repair the corpus and the wording of assignments. Do not grade students for asking. Pair stubborn confusion with an early engagement alert only if your teaching team already treats low activity as a care signal. Chat volume is not mastery. Academic-integrity rules stay in the syllabus you wrote, including whether tutor use is allowed on closed-note exams.

When you change a lecture, replace the file and retire old chunk IDs so the tutor cannot cite a withdrawn claim. When you change a deadline, change the syllabus object the retriever sees. Otherwise the tutor keeps citing Tuesday while the calendar in Canvas, Blackboard, Moodle, or Anthology already says Thursday.

Hold the same outcome in review that you hold at runtime: an answer that cites the course materials used, or a blank when retrieval is empty. A fluent explanation from nowhere, a grade implied in chat, or a policy invented on the student's behalf is the system failing the course.

Is this worth automating for you?

Whether this pays back depends on how much time it takes your team today. Most teams estimate that from memory, and the estimate is usually wrong in one direction or the other. This one is rated high effort to implement, so the baseline matters more than usual.

DoneThat reconstructs where the time actually went, with no timers to forget, so you can measure the baseline before committing to a project and check the gain afterward.

Measure the baseline first