In 1956, when mathematician and theoretical biologist Jack D. Cowan was in his early 20s, he visited Wilfred Taylor and his strange new “teaching machine” on a one-year trip to London. When he arrived, he was stunned by the “wide array of apparatus” in front of him. Cowan could only step aside and watch the “machine do its job.” Apparently what it did was run a “associative memory circuit” – it seemed to be able to learn to find connections and retrieve data.

It might look like a pile of wires and clumsy circuit blocks hand-soldered together in a box, but what Cowan observed was an early analog form of a neural network; the ability to create responsive written content on virtually any team. The underlying technology of ChatGPT is a neural network.

As Cowan and Taylor stood and watched the machine at work, they had no idea how exactly the machine had accomplished this task. The answer to Taylor’s mysterious machine brain can be found in its “analog neurons,” the associations created by machine memory, and most importantly, in the fact that its automatic functioning cannot really be fully explained. It will take decades for these systems to find their purpose and unleash this power.

The term “neural network” encompasses a wide variety of systems, but according to IBM these “neural networks – also known as artificial neural networks (ANNs) or simulated neural networks (SNNs) – are a subset of machine learning and heart of deep learning algorithms.” . More importantly, the term itself, its shape, and “structure are inspired by the human brain, mimicking the way biological neurons transmit signals to one another.”

There may have been some skepticism about their value in the early stages, but over the years, the fashion for artificial intelligence has definitely returned to neural networks. They are now generally considered the future of AI. They are of great importance to us and to what it means to be human. Recently, we’ve heard feedback on these concerns, with calls to suspend new AI development for six months to see what the impact is.

It would certainly be a mistake to dismiss neural networks as shiny, attractive new tools. They came into our lives for good. Some are strong in practicality. In 1989, a team led by Yann LeCun of AT&T Bell Laboratories used backpropagation techniques to teach a system to recognize handwritten zip codes. Microsoft’s recent announcement that Bing search will be powered by AI and making it your “copilot for the web” suggests that what we discover and how we understand it will increasingly be the product of this type of automation.

Artificial intelligence can also be trained to do things like rapid image recognition, relying on huge amounts of data to find patterns, allowing these to be incorporated into facial recognition, for example. The ability to identify patterns has led to many other applications, such as predicting stock markets.

Neural networks are also changing the way we interpret and communicate. Developed by the interestingly named Google Brain Team, Google Translate is another well-known neural network application.

You won’t want to play chess with him either. Their understanding of the rules, memorizing strategies and all recorded moves mean they are extremely good at games (although ChatGPT performs poorly with Wordle). The systems that plague human Go players (Go is a strategic board game known to be difficult) and chess masters are built from neural networks.

But their scope goes far beyond these cases and continues to expand. A patent search limited to specifying the exact phrase “neural networks” yields 135,828 results. With this rapid and continuous expansion, our chances of fully explaining the impact of artificial intelligence may become less and less. These are questions I address in my research and in my new book on algorithmic thinking. Read more…