Live demo: ml-visoft-c-reviewer.hf.space
Mentoring is a time-consuming activity, and when there are many students, the mentor’s attention is
divided. Fairly strict schedules and scarce availability makes the mentoring experience, for the student,
more difficult than it should be. LLMs on the other hand are always available but they lack subject
matter expertise.
I developed a LLM powered code reviewing tool to help first year students gain
clean-code experience. The students can get quick feedback 24/7 and they are not blocked in their
learning, by the next available slot of their teacher.
LLMs are trained on the whole internet and relying on public datasets for performance estimation
comes with a great risk of overfitting.
One of the most difficult tasks was to gather data to evaluate how well the LLM works for our problem.
I designed a synthetic data flywheel that generated labeled samples with minimal human interven-
tion.
The whole system was evaluated using Jaccard score for four clean code principles.
More details on my blog post.