Why Learn About Neural Networks Today?

I posted this on LinkedIn last week and wanted to share it here as well.

Yesterday I gave my annual intro to neural networks lecture to a lovely cohort of MSc students, and this year prepping for it got me thinking about how much the AI/ML space has changed recently.

We can now vibe-code software, websites, and even fairly basic apps. We can use LLMs to act as copilots in development, and even run AI agents that handle operational tasks, usually with some form of a human-in-the-loop.

These models can do a lot, but I posed a question: why bother learning neural networks at a level that requires understanding basic concepts, even with some maths, at a toy problem level?

My answer is simple. It keeps you grounded.

There’s a lot of magical thinking in this space. People anthropomorphise their favourite LLMs or take LLM outputs at face value because they sound plausible, especially in domains where they don’t even have enough experience or expertise to challenge them!

I usually suggest to folks (closer to the field) that if you want a feel for what’s going on under the hood, then train a basic neural network on a small problem yourself. This will force you to face their limitations directly, rather than treating the entire thing like a magical black box.

You’ll see how much effort it takes to train a model from data preparation through to choosing appropriate evaluation metrics. You might end up with a good model, but the inconsistencies will be glaringly obvious.

Now imagine scaling this to the gargantuan models we have today. Even though LLMs aren’t directly comparable (their capabilities are indeed super impressive), the underlying fundamentals in training don’t just disappear. They don’t go away but become harder to see and reason about. Going through this process may give you a much clearer appreciation of that.

It’s also difficult to predict where this space is heading in the next 5 years. There are a lot of conflicting opinions, even among experts. Scaling models may only take you so far, and if that’s the case, then a different paradigm and direction may need to be undertaken.

And how does that happen? When we go back to the drawing board and revisit the basics.