Learning with LLMs

One common frustration when learning from 'set texts' (documentation, manuals, books etc.) is that the author has to try and reach as many people as possible, but has to make assumptions about what the average reader already knows. This means it might assume some knowledge you don't have, or waste time explaining things you already know.

You don't need to do this with LLMs. You can tell it what you are trying to understand, and tell it exactly what you already know and don't know. You are essentially inviting the LLM to frame the explanation in terms of concepts you are already familiar with, while asking it to take its time with concepts that might ordinarily be brushed over.

Different people also respond differently to different kinds of explanations, and it takes different structures of explanation to make things 'click' for different people. Again, you can just tell the LLM this to get an explanation tailored specifically to you.

I am trying to learn about Kubernetes. I am already familiar with the concept of images and containers, and have deployed apps using tools like docker compose. My expertise is primarily in software development generally and web development specifically - I have less experience with complex production environments, or with server hardware generally. I prefer explanations to be high level, and framed in terms of the problems a technology solves.

Sometimes we come to LLMs having already tried to understand something, but for whatever reason the model in our head of how the thing works just doesn't hold together. There's some implication of how you think it works that clashes with something else, or there's a key point that traditional sources tend to gloss over.

In this case, what you need is not a general overview, but a clarification of one specific point - which when given often resolves the whole understanding in your mind.

I am trying to understand how capital allowances work in UK companies. I understand that it allows you to deduct the cost of equipment from your profit, but I thought investment in assets like this was already deductible? If my company brings in £100000, I spend £50000 on producing goods, and then £30000 on machinery, my profit is £20000 - why do I need capital allowances? What am I missing here?

The best way to understand something is almost always to make your own notes as you go, and this hasn't changed in the LLM age. They are useful as a source to refer back to (especially if you have a personal wiki or anything like that) but even if you don't, I find most of the value of these personal notes is generated in the act of writing them down. It is easy to tell yourself that you have understood something, but forcing your brain to express that understanding in words - full, well structured sentences in subsections that follow logically from each other - quickly highlights any gaps.

And this system gels perfectly with an LLM. After the back and forth process of asking questions, clarifying specific points, building up your own summary draft, give your completed notes to an LLM and get it confirm that what you have described is an accurate model of whatever you are trying to learn. I often find that its default behaviour is to just return a rewritten version of the notes which corrections made, which is very unhelpful - specifically tell it you want it to critique, and flag anything which is either incorrect, or indicates something hasn't quite been understood.

One of the flaws of LLMs, at least at time of writing, is that they're very eager to please. They have a sometimes eerily positive tone, can be very easily led, and I've found that if you're not careful with your prompting, the LLM will sometimes pick up on what the answer you're expecting is and lean in that direction - sometimes quite subtly.

This can have an impact on the methodology we're developing here too. Usually the above method of feeding it your own summary back at it works fine, but there's always that small doubt - is it just telling me what I want to hear?

The solution is simple. Give it your notes as a summary, but change one key fact or explanation in it to something you know to be incorrect - at least as far as your current understanding goes. If you have really understood what you have made notes on, and if the LLM is really being brutally honest and truthful, then it will immediately hone in on the mistake you've introduced and flag that as wrong, while approving of the rest.

Once you're finally sure your understanding is correct, a useful final confirmatory step can be to start a fresh context and present your notes there. There is a danger that in the previous context, the LLM was being influenced by the conversation leading up to the final state into being more agreeable to you than possibly it should. Conversations in which one party progressively builds up their understanding of something tend to end with the other party confirming understanding, and we don't want the LLM just conforming to the pattern of how these usually go, even slightly.

Starting from scratch, presenting your understanding to the LLM which knows nothing of this previous iterative process, and getting its approval can be very reassuring. The highest form of understanding is the ability to explain the concept from first principles to someone, and that holds true when then 'someone' is an LLM with a fresh context.