Prepared by: Lama R. Khreiss

Advanced Quantitative Methods in Business – MGT 501
Neural Network Technique

Outline

* Overview ………………………………………………………….……… 4 * Definition …………………………………………………4 * The Basics of Neural Networks……………………………………………5 * Major Components of an Artificial Neuron………………………………..5 * Applications of Neural Networks ……………….9 * Advantages and Disadvantages of Neural Networks……………………...12 * Example……………………………………………………………………14 * Conclusion …………………………………………………………………14

Overview

One of the most crucial and dominant subjects in management studies is finding more effective tools for complicated managerial problems, and due to the advancement of computer and communication technology, tools used in management decisions have undergone a massive change. Artificial Neural Networks (ANNs) is an example, knowing that it has become a critical component of business intelligence. The below article describes the basics of neural networks as well as some work done on the application of ANNs in management sciences.

Definition of a Neural Network?
The simplest definition of a neural network, particularly referred to as an 'artificial' neural network (ANN), is provided by the inventor of one of the first neurocomputers, Dr. Robert Hecht-Nielsen who defines a neural network as follows:
"...a computing system made up of a number of simple, highly interconnected processing elements, which process information by their dynamic state response to external inputs."Neural Network Primer: Part I" by Maureen Caudill, AI Expert, Feb. 1989.

ANNs are processing devices (algorithms or actual hardware) that are loosely modeled after the neuronal structure of the mamalian cerebral cortex but on much smaller scales. A large ANN might have hundreds or thousands of processor units, whereas a mamalian brain has billions of neurons with a corresponding increase in magnitude of their overall interaction and emergent behavior. Although ANN researchers are generally not concerned with whether their networks accurately resemble biological systems, yet some are. For example, researchers have accurately simulated the function of the retina and modeled the eye rather well.
Despite that the mathematics involved with neural networking is not a trivial matter, a user can rather easily gain at least an operational understanding of their structure and function.
A neural network is an information processing system that is non-algorithmic , non – digital , and intensely parallel . It is not a computer, nor it is programmed like one . Instead , it consists of a number of very simple and highly interconnected processes called artificial neurons that represent analogs of the biological neural cells , or neurons. In the brain , the neurons are connected by a large number of weighted links , over which signals can pass. Each neuron typically receives many signals over its incoming connections ; some of these incoming signals may arise from other neurons , and others come from the outside world .The neuron usually has many of these incoming signal connections; however , it never produces more than a single outgoing signal .
The Basics of Neural Networks
Neural neworks are typically organized in layers. Layers are made up of a number of interconnected 'nodes' which contain an 'activation function'. Patterns are presented to the network via the 'input layer', which communicates to one or more 'hidden layers' where the actual processing is done via a system of weighted 'connections'. The hidden layers then link to an 'output layer' where the answer is output as shown in the graphic below.

.
Most ANNs contain some form of 'learning rule' which modifies the weights of the connections according to the input patterns that are presented with it. That is, ANNs learn by example as do their biological counterparts; a child learns to recognize dogs from examples of dogs.
Although there are many different kinds of learning rules used by neural networks, this demonstration is concerned only with one; the delta rule. The delta rule is often utilized by the most common class of ANNs called 'backpropagational neural networks' (BPNNs). Backpropagation is an abbreviation for the backwards propagation of error.

Major Components of an Artificial Neuron
This section describes the seven major components which make up an artificial neuron. These components are valid whether the neuron is used for input, output, or is in one of the hidden layers.

Component 1: Weighting Factors: A neuron usually receives many simultaneous inputs. Each input has its own relative weight which gives the input the impact that it needs on the processing element's summation function. These weights perform the same type of function as the varying synaptic strengths of biological neurons. In both cases, some inputs are made more important than others in order to have a greater effect on the processing element as they combine to produce a neural response.
Weights are adaptive coefficients within the network that determine the intensity of the input signal as registered by the artificial neuron. They are a measure of an input's connection strength. These strengths can be modified in response to various training sets and according to a network's specific topology or through its learning rules.
Component 2: Summation Function: The first step in processing an element's operation is to compute the weighted sum of all of the inputs. Mathematically, the inputs and the corresponding weights are vectors which can be represented as (i1, i2 . . . in) and (w1, w2 . . . wn). The total input signal is the dot, or inner, product of these two vectors. This simplistic summation function is found by multiplying each component of the i vector by the corresponding component of the w vector and then adding up all the products. Input1 = i1 * w1, input2 = i2 * w2, etc., are added as input1 + input2 + . . . + inputn. The result is a single number, not a multi-element vector.
Geometrically, the inner product of the two vectors can be considered a measure of their similarity. If the vectors point in the same direction, the inner product is maximum; if the vectors point in opposite direction (180 degrees out of phase), the inner product is minimum.
The summation function can be more complex than just the simple input and weight sum of products. The input and weighting coefficients can be combined in many different ways before passing on to the transfer function. In addition to a simple product summing, the summation function can select the minimum, maximum, majority, product, or several normalizing algorithms. The specific algorithm for combining neural inputs is determined by the chosen network architecture and paradigm.
Some summation functions have an additional process applied to the result before it is passed on to the transfer function. This process is sometimes called the activation function. The purpose of utilizing an activation function is to allow the summation output to vary with respect to time. Activation functions currently are pretty much confined to research. Most of the current network implementations use an "identity" activation function, which is equivalent to not having one. Moreover, such a function is likely to be a component of the network as a whole rather than of each individual processing element component.

Component 3:
Transfer Function: The result of the summation function, that’s almost always the weighted sum, is transformed to a working output through an algorithmic process known as the transfer function. In the transfer function the summation total can be compared with some threshold to determine the neural output. If the sum is greater than the threshold value, the processing element generates a signal. If the sum of the input and weight products is less than the threshold, no signal (or some inhibitory signal) is generated. Both types of response are significant.
The threshold, or transfer function, is generally non-linear. Linear (straight-line) functions are limited because the output is simply proportional to the input. However, linear functions are not very useful, and this was the problem in the earliest network models as noted in Minsky and Papert's book Perceptrons.
The transfer function could be something as simple as depending upon whether the result of the summation function is positive or negative. The network could output zero and one, one and minus one, or other numeric combinations. The transfer function would then be a "hard limiter" or step function.
Another type of transfer function, the threshold or ramping function could mirror the input within a given range and still act as a hard limiter outside that range. It is a linear function that has been clipped to minimum and maximum values, making it non-linear. Yet another option would be a sigmoid or S-shaped curve. That curve approaches a minimum and maximum value at the asymptotes. It is common for this curve to be called a sigmoid when it ranges between 0 and 1, and a hyperbolic tangent when it ranges between -1 and 1. Mathematically, the exciting feature of these curves is that both the function and its derivatives are continuous. This option works fairly well and is often the transfer function of choice. Other transfer functions are dedicated to specific network architectures and shall be discussed later in the article.
Prior to applying the transfer function, uniformly distributed random noise may be added. The source and amount of this noise is determined by the learning mode of a given network paradigm. This noise is normally referred to as "temperature" of the artificial neurons. The name, temperature, is derived from the physical phenomenon which explains that as people becomes too hot or cold their thinking abilities are affected. Electronically, this process is simulated by adding noise. Indeed, by adding different levels of noise to the summation result, more brain-like transfer functions are realized. To more closely mimic nature's characteristics, some experimenters use Gaussian noise source. Gaussian noise is similar to a uniformly distributed noise except that the distribution of random numbers within the temperature range is along a bell curve. The use of temperature is an ongoing research area and is not being applied to many engineering applications.
NASA announced a network topology which uses what it calls a temperature coefficient in a new feed-forward, back-propagation learning function. But this temperature coefficient is a global term which is applied to the gain of the transfer function, and is different from the more common term, temperature, which is simple noise being added to individual neurons. In contrast, the global temperature coefficient allows the transfer function to have a learning variable much like the synaptic input weights. This concept is claimed to create a network which has a significantly faster (by several order of magnitudes) learning rate and provides more accurate results than other feed forward, back-propagation networks.
Component 4:
Scaling and Limiting: After processing the element's transfer function, the result can pass through additional processes which scale and limit. This scaling simply multiplies a scale factor by the transfer value, and then adds an offset. Limiting is the mechanism which insures that the scaled result does not exceed an upper or lower bound. This limiting is in addition to the hard limits that the original transfer function may have performed.
This type of scaling and limiting is mainly used in topologies to test biological neuron models, such as James Anderson's brain-state-in-the-box.
Component 5:
Output Function (Competition): Each processing element is allowed one output signal which it may output to hundreds of other neurons. This is just like the biological neuron, where there are many inputs and only one output action. Normally, the output is directly equivalent to the transfer function's result. Some network topologies, however, modify the transfer result to incorporate competition among neighboring processing elements. Neurons are allowed to compete with each other, inhibiting processing elements unless they have great strength. Competition can occur at one or both of two levels. First, competition determines which artificial neuron will be active, or provides an output. Second, competitive inputs help determine which processing element will participate in the learning or adaptation process.
Component 6:
Error Function and Back-Propagated Value: In most learning networks the difference between the current output and the desired output is calculated. This raw error is then transformed by the error function to match particular network architecture. The most basic architectures use this error directly, but some square the error while retaining its sign, some cube the error, and other paradigms modify the raw error to fit their specific purposes. The artificial neuron's error is then typically propagated into the learning function of another processing element. This error term is sometimes called the current error.
The current error is typically propagated backwards to a previous layer. Yet, this back-propagated value can be either the current error, the current error scaled in some manner (often by the derivative of the transfer function), or some other desired output depending on the network type. Normally, this back-propagated value, after being scaled by the learning function, is multiplied against each of the incoming connection weights to modify them before the next learning cycle.

Component 7:
Learning Function: The purpose of the learning function is to modify the variable connection of weights on the inputs of each processing element according to some neural based algorithm. This process of changing the weights of the input connections to achieve some desired result can also be called the adaption function, as well as the learning mode. There are two types of learning: supervised and unsupervised. Supervised learning requires a teacher. The teacher may be a training set of data or an observer who grades the performance of the network results. Either way, having a teacher is learning by reinforcement. When there is no external teacher, the system must organize itself by some internal criteria designed into the network. This is learning by doing.

Applications of Neural Networks
Neural networks are universal approximators, and they work best if the system that’s being used to model them has a high tolerance to error. One would therefore not be advised to use a neural network to balance one's cheque book! However they work very well when: * Capturing associations or discovering regularities within a set of patterns; * The volume, number of variables or diversity of the data is very great; * The relationships between variables are vaguely understood; or, * The relationships are difficult to describe adequately with conventional approaches.
Application of Artificial neural networks :
Artificial neural networks are applied in different aspects of business sciences such as marketing, finance , manufacturing and strategic management .
ANN can be applied in many marketing decision making problems which could be done previously by multivariate statistical analysis only. Typical problems turn out to be market segmentation tasks and classification of consumer spending patterns , new product analysis ,identification of customer characteristics , sale forecasts ,targeted marketing , and modeling the relationship between market orientation and performance ,
The critical point for research in the field of marketing is the lack of applications on the individual level data which is a problem encountered in ANN applications .
ANN application in finance : ANNs are now frequently used in many modeling and forecasting problems. The main advantage of this tool is the ability to approximate almost any nonlinear function arbitrarily close . The ANN can provide a better fit compared with parametric linear models. On the other hand it’s difficult to interpret the meaning of parameters. The essential topics in finance are forecasts of changes in the value of financial assets under the form of stocks , indexes and currencies , analysis of financial statements .
ANN application in manufacturing and production :
ANN can be used in forecasting (production costs , delivery dates , etc.),quality control and optimization that are predominant in production problems . As quality control problems correspond to classification. The appropriateness of ANN approaches is supposed to be as good in fields of marketing and finance .
ANN applied with the field of strategic planning and business policy: The empirical research in strategic planning systems has focused on two areas : the impact of strategic planning on firm performance and the role of strategic planning in strategic decision making .

Table 1: reported applications in the field of marketing and sales Business area | Problem type | | | Marketing and sales | Forecasting costumer response Sales forecast Target marketing Market segmentation Brand analysis Storage layoutCustomer gender analysisMarketing data mining New product acceptance research Consumer choice prediction Market share forecasting | | |

Table 2: reported applications in the field of finance and accounting

Business area | Problem type | | | Finance and accounting | Financial health predictionBankruptcy classificationAnalytical review processCredit scoringRisk assessmentForecastingBond ratingMutual fund selectionCredit evaluation | | |

Table 3: reported applications in the field of manufacturing and production Business area | Problem type | | | | | | | manufacturing and production | Engineering designQuality controlStorage designInventory controlSupply chain managementDemand forecastingProcess selection | | |

Table 4: reported applications in the field of strategic management and business policy

Business area | Problem type | | | | | | | strategic management and business policy | Strategic performance and planningAssessing decision makingEvaluating strategies | | |

Advantages and Disadvantages of Neural Networks:
The advantage of neural networks over conventional programming lies on their ability to solve problems that do not have an algorithmic solution or when the available solution is too complex to be found. Neural networks are well suited to tackle problems that people are good at solving, such as prediction and pattern recognition. Neural networks have been applied within the medical domain for clinical diagnosis, image analysis and interpretation, signal analysis and interpretation, and drug development. The classification of the applications presented below is simplified, since most of the examples lie in more than one category (e.g. diagnosis and image interpretation; diagnosis and signal interpretation). Depending on the nature of the application and the strength of the internal data patterns you can generally expect a network to train quite well. This applies to problems where the relationships may be quite dynamic or non-linear. ANNs provide an analytical alternative to conventional techniques which are often limited by strict assumptions of normality, linearity, variable independence etc. Because an ANN can capture many kinds of relationships it allows the user to quickly and relatively easily model phenomena which otherwise may have been very difficult or impossible to explain otherwise.

Neural networks almost always underperform approaches that have stronger statistical foundations, and tend to be more difficult to work with, for the following reasons:

1. Neural networks are not magic hammers. Neural networks are still viewed by many as being magic hammers that can solve any machine learning problem, and as a result people tend to apply them indiscriminately to problems for which they are not well suited. Although neural networks do have a proven track record of success for certain specific problems, as a consumer of machine learning technology you're almost always better off using approaches that have stronger theoretical underpinnings. Rather than just throwing a general-purpose neural network at your problem and hoping for the best, try to understand and simplify your problem as well as possible, and look for techniques that have a structure which will fit the task well. 2. Neural networks are too much of a black box. This makes them difficult to train; the training outcome can be nondeterministic and depend crucially on the choice of initial parameters, e.g. the starting point for gradient descent when training back propagation networks; and their opaque nature makes it very hard to determine how they are solving a problem. They are difficult to troubleshoot when they don't work, and when they do work, you never really feel confident that they will generalize well to data not included in your training set because, fundamentally, you don't understand what your network is doing. 3. Neural networks are not probabilistic. For the most part, NNs have few if any probabilistic underpinnings, unlike their more statistical or Bayesian counterparts in the more general field of machine learning. It's incredibly useful to know how confident your classifier is about its answers, because that information allows you to better manage the cost of making errors. A neural network might give you a continuous number as its output (e.g. a score) but translating that into a probability is hard. Approaches with stronger theoretical foundations tend to give you those probabilities directly. 4. Neural networks are not a substitute for understanding problems: Rather than using a neural net, you're almost always better off investing a little extra time studying, analyzing and dissecting data first, then choosing some better grounded technique that you know will work well for the problem. For example, if you are building a classifier, rather than just throwing a neural network at your data and hoping for the best, spend time visualizing the dataset and selecting or creating the best input features using whatever domain-specific knowledge and expertise available to you. You might discover that you can clearly differentiate between your training classes using just three out of your original ten features, or that you need to develop a more sophisticated nonlinear derived feature to better separate your classes. Ultimately for example, you might end up using a Gaussian mixture model (GMM) to directly model the density function of your classes, which will allow you to use Bayes theorem to deduce class probabilities and give you a far better classifier than you could ever build using a neural network.

Advantages:
· A neural network can perform tasks that a linear program cannot.
· When an element of the neural network fails, it can continue without any problem since it’s parallel by nature.
· A neural network learns and does not need to be reprogrammed.
· It can be implemented in any application and without any problem.
Disadvantages:
· The neural network needs training to operate.
· The architecture of a neural network is different from the architecture of microprocessors therefore needs to be emulated.
· Requires high processing time for large neural networks.

Examples:

* Fraudulent credit card detection (VISA) * Speech recognition, text-to-speech (NetTalk) * Price share prediction (time series analysis) * Image compression

Conclusion:
By studying artificial Neural Network it can be concluded that since technology is developing day by day the need of Artificial Intelligence is increasing because of only parallel processing. Parallel Processing is more needed in this present time because it saves more time and money in any work related to computers and robots. Considering future work one can only say that more algorithms, and other problem solving techniques should be developed so that limitations of Artificial Neural Network are removed. In addition, Artificial Neural Network used to learn to predict future events based on the patterns that have been observed in the historical training data and learn to classify unseen data into pre-defined groups based on characteristics observed in the training data. Artificial neural networks are applications in management , marketing , financing , applied with the field of manufacturing and production , to illustrate that a properly designed ANN is an interactive software – based system intended to help decision makers compile useful information from raw data , documents ,personal knowledge , and/ or business models to identify ,solve problems , and make decisions .

References:
Hejase , H. J., & Hejase , A. J. (2011). management information systems . (first ed., pp. 147-148). Beirut: Dar Sader.
White, H., 1989. Neural Network Learning and Statistics. AI Expert, 4(12): 48-82.
Wang, S., An adaptive approach to market development forecasting. Neural Computing and Applications, 1999. 8(1): p. 3-8.
Vellido, A., P.J.G. Lisboa and K. Meehan, 1999. Segmentation of the online shopping market using neural networks. Expert Systems with Applications, 17(4): 303-314.