By Matt Tank
Machine learning is an integral part of modern artificial intelligence, but what is it exactly? In a nutshell, it’s the process by which an AI system increases its capability, or “learns”. This could mean several things, such as an improved chess game, more accurately predicting stock market fluctuations, or becoming better at reading human emotion.
Data is the currency of machine learning – The more data that’s available to the system, the better the system will learn. This data can be created in a number of ways. For example, early chess AIs were given information from historical chess matches, and built their knowledge from this data. In comparison, Google’s DeepMind AlphaZero system was given only the rules of chess. It built the data it needed by playing simulated games against itself. 68 million simulated games over four hours was all it needed to beat the current crop of chess-playing AIs.
The principles of machine learning are quite a bit different to those of traditional software development or data analysis. As an example, when building a system that prevents email spam, you would traditionally program in rules about how to recognise a spam email. This is not as easy as it sounds, and to top it off, when spammers’ techniques change, you have to write in new rules.
When using machine learning however, you approach it differently. Once you create the machine learning system, which is essentially a program designed to detect patterns in data, you have to train it. You do this by providing the data to be looked at, along with the answer you’re trying to find. In this example, you feed in copies of the email, along with the result, which is “has someone flagged this as spam?”.
With this information, the system can then look for patterns in the emails, and work out whether those patterns help identify the email as spam. Some of these patterns are obvious, like “is the sending email address valid?”. Others might be so specific, or hard to detect that a human could never put their finger on it, like “an email with language patterns that match Nigerian English with a PDF attachment of 20-100KB is 98% likely to be spam”.
Once the system has been trained, the “model”, representing the knowledge it has gained, can be used to actually detect spam. However, in most cases, the training never really stops, so every time you mark an email as spam, or if you mark an email in your Junk Mail folder as “not spam”, it is helping to train the machine learning system and improve its capability.
The key expert for working with machine learning systems is the data scientist. This person is responsible for making sure the right data is fed into the system. The data scientist then looks at the result of the training, and determines if any more data is required, or if irrelevant data is interfering with the results. This will usually require a number of repetitions and tweaks to get right.
The speed in which computers can process data, and refine their performance, and their ability to detect patterns that humans can’t, means that machine learning will have a huge impact going forward. The applications are endless, from curing disease, to predicting natural disasters, to understanding human language, and there is unlikely to be any area of our lives that won’t be affected. As such, understanding how these systems work will be an important part of many jobs in the near future.