NASHVILLE, Tenn. ― Digital Reasoning, a leader in cognitive computing, today announces it has trained the largest neural network in the world to date with a stunning 160 billion parameters.
Google’s previous record was 11.2 billion, and Lawrence Livermore National Laboratory recently trained a neural network with 15 billion parameters.
The results of Digital Reasoning’s research with deep learning and neural networks is published in the Journal of Machine Learning and Arxiv alongside other notable companies like Google, Facebook and Microsoft and will be presented July 6-11 at the prestigious 32nd International Conference on Machine Learning in Lille, France.
Neural Networks are computer systems modeled after the human brain. Like the human brain, these networks can gather new data, process it, and react to it. Digital Reasoning’s paper, titled “Modeling Order in Neural Word Embeddings at Scale,” details both the impressive scope of their neural network as well as the exponential improvement in quality.
In their research, Matthew Russell, Digital Reasoning’s Chief Technology Officer, and his team evaluated neural word embeddings on “word analogy” accuracy. Neural networks generate a vector of numbers for each word in a vocabulary. This allowed the Digital Reasoning team to do “word math.” For instance “king” minus “man” plus “woman” would yield a result of “queen.” There is an industry standard dataset
of around 20,000 word analogies.
Google’s previous accuracy on this metric was a 76.2 percent accuracy rate. In other words, Google was able to get 76.2 percent of the word analogies “correct” in their system. Stanford’s best score is a 75.0 percent accuracy. Digital Reasoning’s model achieves a score of 85.8 percent accuracy, which is a near 40 percent reduction in error over both Google and Stanford. This is a massive advancement in the state of the art.
“We are extremely proud of the results we have achieved, and the contribution we are making daily to the field of deep learning. This is a tremendous accomplishment for the company and marks an important milestone in putting a defensible stake in the ground towards our position as not just a thought leader in the space, but as an organization that is truly advancing the state of the art in a rigorous peer reviewed way,” Russell said.
About Digital Reasoning
Digital Reasoning delivers cognitive computing for a better world. For more information, visit http://www.