Deep Learning Myths, Lies, and Videotape - Part 1: Verities!
Over the history of IT there always seem to be certain high-profile popular terms and concepts that dominate discussion in different eras and capture the imagination, often filling the vocabulary of IT people in those eras. I believe that in recent years one such subject is Artificial Intelligence (AI). Unlike most popular IT terms that come and go, AI has shown that it really has some serious legs, by extending it’s reign of popularity with two additional hot terms associated with its derivative forms: Machine Learning (ML) and Deep Learning (DL).
Alan Turing was the first to discuss the concept of “intelligence” relative to computers in his famous 1950 paper, "Computing Machinery and Intelligence," which opens with the question, “Can machines think?” and where he introduces The Turing Test (aka: The Imitation Game).
In my opinion, and as a computer scientist that fancies himself a student of the history of computer science (and in my deep past has written applications in LISP and Algol that my professors called “AI”), I believe that in recent years the terms AI, ML, and DL are often misused by IT vendors rather than applied accurately. This seems to be particularly true when it comes to marketing storage products. So let’s separate the truth from the myths and the lies, and start with some definitions.
The foundational definition for the entire subject was established by Alan Turing himself: “If a human could not distinguish between responses from a machine and a human, the machine could be considered intelligent.”
The term artificial intelligence was first coined by professor John McCarthy in 1956, where he defined it as “Getting a computer to do things which, when done by people, are said to involve intelligence.”
The term machine learning was introduced by Arthur Samuel in 1959 while at IBM, where he developed the first intelligent computer game (checkers). Machine learning is a direct subset of AI. All ML is also AI, but not all AI is ML. Samuel's definition is: “Machine learning is an application of artificial intelligence (AI) that provides systems the ability to automatically learn and improve from experience without being explicitly programmed.”
The fundamental difference between AI and ML is that while an AI function may demonstrate intelligence, all of that intelligence was directed explicitly by humans that defined the programming. In ML, the computer itself is capable of making new and independent decisions based upon new input and its programming, without additional human directive.
Deep learning is a direct subset of ML. So all DL is also ML, but not all ML is DL. Professor Rena Dechter introduced the term in 1986, and defined it as: “Deep learning is a class of machine learning algorithms that use multiple layers to progressively extract higher level features from raw input.”
The basic difference between DL and ML is that DL analyzes multiple options at multiple times when making decisions. DL then analyzes, tests, and learns from the effectiveness of its previous decisions to continually and constantly improve the quality of its future decisions. DL is clearly the most intensive and complex approach of the three forms of AI, but also the one that has the most potential to provide the best and most effective results.
If something claims to be AI, ML, or DL, it follows that the “acid test” should be how well it meets the definitions described above.
DL in Enterprise Storage today: InfiniBox
Infinidat’s patented Neural Cache software is the “secret sauce” behind how our InfiniBox storage system is able to get faster than all-flash array performance at revolutionary lower cost than alternative all-flash arrays. It does this by offering the highest cache-hit ratios of any storage system in the history of the market (by a lot!) and thus being able to use low-cost media as its highly-protected persistent storage.
When I joined Infinidat in 2019, I noticed that Neural Cache was described as “machine learning” in all the materials that I read. As I went deeper and learned more about it, I agreed that it met the definitions for both AI (the basic test to pass) and ML (a harder test to pass). But I also felt that we were not giving Neural Cache the higher level of respect that it deserved — as a true DL solution.
My case to justify Neural Cache as DL is simple. In the rare (typically on average only 5-7% of the time) situation in which data that has been requested by an application server is not already in the cache of an InfiniBox, it must then be retrieved from the back-end persistent storage. In addition to bringing the requested data into cache to deliver it to the application server, Neural Cache software will also bring in multiple other extra pieces of data into cache. These additional pieces of data will be cached based upon previous intelligent analysis of their possible associations with the piece of data that actually was requested by the application. The Neural Cache software will then observe, analyze, and learn which of those extra pieces of data (if any) were subsequently also requested, and when. That learning will then influence the future multiple options for extra data that might be brought into cache prior to an actual request from an application.
This is true DL! The benefit for users is faster performance than all-flash arrays which derive their performance attributes from hardware components, and at a radically lower cost. This is the advantage of math over media, or software over hardware. It is uniquely enabled by DL.
In Part 2 of this blog post, we’ll take a look at what other storage systems that claim to use AI are doing!