March 11, 1996 Issue 892, page 38
Redmond, Wash. - Software that uses Darwinian principles to search through hundreds of neural networks for the best candidate for a particular application has just come to market.
Based on genetic algorithms, the technology, called NeuroGenesis, comes from BioComp, a contract consulting company here. It suits a number of application areas, such as modeling, classification, diagnostics or time-series predictions, says the company, which is also designing a high-end server that will allow users to easily manage hundreds of NeuroGenesis networks.
"As a consulting company, we just got tired of having to adjust all the parameters on these commercial neural-network packages," said Carl Cook, president of BioComp. "For a 20-input neural network the number of possible combinations of neurons, connections, layers and transfer functions is more than 100 billion. So we decided to get a genetic algorithm to try out different combinations for us."
The heart of the resulting software, the NeuroGenetic Optimizer™ (NGO), tries out hundreds of alternative neural architectures and configurations using the principles of natural selection. NGO creates and evaluates a population of possible networks and goes through them, generation after generation, until it finds an optimal one.
Even though the NGO evaluates hundreds of neural nets, it doesn't take a hundred-fold increase in processing time to do so. "It depends, of course, on many factors about the neural architecture as well as the training data, but as a rule of thumb it usually takes from a few hours to overnight," said Cook.
First try
On an overnight job, the NGO might evaluate as many as 1,500 alternative neural networks, any one of which could later be judged the best. "Many times NGO's first try is the best one-because the NGO always tries the conventional, fully connected neural network on its first try," Cook said. "But when it's not the first one, we often find a big improvement, because the genetic algorithm has set many parameters that it would have taken too much labor to set manually."
Once the NGO development tool has created a neural network, it can be run in other environments by virtue of the NeuroGenesis line members. The Penney tool permits the network to be exported and run within popular spreadsheets. ExamiNeur™ can load and run the NGO network to unravel the learning inside their connection weights; it presents that knowledge in both graphical and tabular formats.
The NGO kernel is also available separately as the GALibrary™, thereby permitting programmers to use its genetic algorithms in other applications. A developer's application programming interface simplifies integrating with other code.
During its extended run times, the NGO displays its status with graphical and tabular views of the evolving population of neural networks, their configurations and statistics regarding the top 10 possibilities. These include their learning curves, and current outputs as compared to the desired training values.
The basic development cycle for an evaluation begins by loading data into main memory, building and validating the training and test data sets, creating an initial population of candidate input variables and neural structures. The NGO then begins building and training the selected neural networks. When training stops because no further improvement in accuracy has been observed, the NGO compares the finished network against the population.
After all members of the population have been built and evaluated, the top contenders, by accuracy, are permitted to exchange genetic material. "We keep the best, throw away the worst, and randomly replace the worst with clones of the best," Cook said. "Plus we throw in some mutations." The top contenders and their genetically mixed and mutated brethren are then run through another round of evaluation, and the cycle repeats.
Once the NGO has settled on a final neural network, it may be embedded in a spreadsheet via the Penney tool or examined in even more detail with ExamiNeur™. Both programs support a bi-directional development effort, because data from them may be sent back to the NGO.
When an NGO-created neural network is loaded into a spreadsheet, "what-if" operations can be performed to see the various influences on outputs of particular input variables. The results of these effects can also be displayed with 3-D graphics.
ExamiNeur™, too, traces the relationship between input and output variables with 3-D visualization tools. ExamiNeur™ also graphically displays predicted vs. desired results. For instance, an engineer might use ExamiNeur™ to find alternative operating conditions that solve a quality-control problem, such as turning up the temperature. Feedback from ExamiNeur™ is in plain English, and no understanding of neural networks is necessary.
Among the many analysis tools within ExamiNeur™ is TargetSeeker™, which finds sets of neural-network input values (within user-specified constraints) that generate a desired set of outputs. "We will be adding a lot of new features to ExamiNeur™-it really helps to understand what a neural network is doing," Cook said.
For engineers set on rolling their own programs and examination tools, the GALibrary™ encapsulates the 13 most often used genetic-algorithm operations into a dynamically linked library written in C for easy porting to other platforms.
"The NGO uses the GALibrary™ to select inputs and evolve the neural-network structures, while ExamiNeur™ uses the GALibrary™ in its TargetSeeker™ function," Cook said. The developer's API is available for programmers wishing to interface NGO directly with their own code.
The server BioComp is designing, called the Enterprise Modeling Server™, will drive a whole network of computers running different NGO neural networks. It allows the user to select the models to be run by priority, then collects the data, runs the neural networks and posts the prediction to the user.
"Say a funds manager wants to know Intel's price next week and Microsoft's price a month from now and a thousand other projections," Cook said. "He or she could train up a neural network for each job and manage them all with the Enterprise Model Server™." The server checks the models periodically for accuracy and regenerates neural networks to replace those that have begun to perform poorly.
The Enterprise Modeling Server™ grew out of a consulting job for a large corporation with hundreds of products, each of which needed a separate neural network for sales forecasting. Using the BioComp server, a single manager at a central site could manage and maintain the hundreds of networks required.
Copyright 1996 by CMP Publications. All rights reserved.