Neural Network Data Mining Explained
Neural network data mining uses artificial neural networks, which are mathematical algorithms aimed at mimicking the way neurons work in our nervous system. They are in essence large curve fitting algorithms, adjusting equations until the prediction matches with reality. For this reason it is very easy to over-train them.
They are used in all fields of endeavor – from trading systems to health diagnostics, and of course they have application in business, particularly when we want to predict a value.
Neural networks are used for prediction and classification, and through the development of self-organizing maps (SOM), for clustering. They are called neural networks because they supposedly mimic the behavior of neurons in the nervous system, taking inputs from the environment, processing them and creating an output. And just in the same way that neurons are linked together, so are nodes in a neural network. As with other data mining techniques neural networks demand that a good selection of relevant inputs are available, that the target output is well understood and that copious amounts of data are available for training.
The most commonly used type of neural network is called a feed forward network. As the name suggests it works by feeding the outputs from each node forward to the next node as its inputs. The flow is essentially one direction, although something called back propagation is used to tune the network by comparing the network’s estimate of a value against the actual value. Nodes in a network do two things. They combine the inputs by multiplying each input by a weight (to simulate its importance) and summing the products – this is called the combination function. Other functions are used, but this is the most common. Secondly, the output from the combination function (a single number) is fed into a transfer function which usually takes the form of a sigmoid (an S shaped curve) or a hyperbolic tangent. These curves allow the network to deal with non-linear behavior. In essence they create a linear relationship for small values, but flatten out for large values. This form of non-linearity is an assumption – but it often works well. The output from the transfer function is then fed to the next node in the network.
Most neural networks have three layers – the input layer, a hidden layer, and the output layer. The hidden layer is so named because it is invisible, with no direct contact to inputs or outputs. Knowing how large to make the hidden layer is one of the crucial issues in using a neural network. Make it too large and the network will simply memorize the training set with absolutely no predictive capability at all. Make it too small and useful patterns will be missed.
Using a neural network requires a considerable amount of skill and the results can range from the sublime to the ridiculous simply by modifying any one of a number of parameters. The most important parameters include:
- The size of the training set.
- The number of hidden layers and the number of nodes in each hidden layer.
- Parameters affecting how quickly the network learns.
- The features to use as input.
- The combination functions and transfer functions.
This is by no means exhaustive and issues such as normalizing inputs, converting categorical inputs and so on, all have a profound effect on the quality of the network produced. Some of the plug and play analytics tools omit neural networks altogether, and for good reason. Other methods produce equally good results without the temperamental behavior. Having said this, neural networks can detect patterns that evade detection by other means, and they are very good at picking up some non-linear behaviors.
Here are 4 tools that can be used in Excel:
This product is easy to use but comes with some advanced features. These include more than 2 output nodes (most other neural networks have just a single node), graphing capabilities, pre-processing and data scaling, and other options for architecture specification and training modes. More advanced than some other Excel add-in alternatives. Very low price of US$ 36 and this includes other tools and various manuals/how-to books.
Palisade provide a number of decision support tools and NeuralTools6 is their neural network offering. It provides a great deal of functionality including training sensitivity analysis to establish the reliability of models. Setting up training, testing and prediction is pretty well automatic and accommodates data spanning multiple worksheets. The neural network manager allows the inevitable large collections of neural networks to be managed easily and both categorical and numerical data are automatically identified. This is an easy to use tool for both end users and analysts. User license costs GBP£ 695 for the Industrial version and GBP£ 495 for the Professional version with a 1000 cases per dataset limit.
NeuroSolutions provide several neural network products and the Excel add-in is both simple to use, but has advanced features for those who need them. The ‘Leave-N-Out’ training lets users omit different data sections in each training run. The result is more robust models. It also features and express builder that automatically builds, trains and tests multiple neural network topologies and reports on the best performing models. Licences cost US$295.
One of several tools provided by Olsoft, NeuroXL Predictor is very easy to use with simple selection of training data and test data. A small number of parameters can be tuned (training rate for example) if necessary, but the product suggests suitable values. Licenses cost US$ 99.95.