Input & Output Scaling

Neural networks learn best when every input arrives on a similar, modest numeric scale. An input that ranges over thousands will swamp one that ranges between 0 and 1, and the network will struggle to weigh them fairly. To prevent this, the toolbox automatically scales your inputs and your target into a small internal range before training, and unscales the prediction back to real units afterwards. You don't have to do anything for this to happen — but understanding it, and the one piece of preparation it can't do for you, makes a real difference to results.

Why scaling matters

Inside the network, activation functions and weight updates behave well only over a limited range of values. Scaling brings every input into that range so they all start on an equal footing, which makes training faster and more stable. The toolbox computes the scaling automatically from your training data — the only choice you make is how it is computed.

The two scaling algorithms: SetScalingAlgorithm

Setting the target range manually

Both algorithms map your data into a target range. By default inputs are mapped into 0..1 and the output into 0..1. You can override these ranges, which is occasionally useful — for example mapping into −1..1 to suit a tanh hidden layer, or matching the output range to your output activation.

Both values must lie between −2 and 2, and Min must be strictly less than Max; out-of-range calls are ignored with a trace message.

SetScalingAlgorithm( 0 );            // Mean / StdDev (the default, shown for clarity)
SetInputScalingMinMax( -1, 1 );     // scale inputs into -1..1 (suits tanh hidden layers)
SetOutputScalingMinMax( -1, 1 );    // scale the target into -1..1

Weighting data points: AddNeuralNetworkAmplify

When you build a training set with the multi-input API, you can attach an amplify channel that tells the network how many times to repeat each data point. A point with an amplify value of 3 is fed to the network three times; a value of 1 (the default) feeds it once; a value of 0 drops it. This lets you give more weight to data points you consider more important — recent bars, or bars that meet some condition — without changing the data itself.

ClearNeuralNetworkInputs();
AddNeuralNetworkInput( PercentDifference( Close, 1 ), 0 );
AddNeuralNetworkOutput( Ref( PercentDifference( Close, 1 ), 1 ), 0 );
// Weight the most recent 200 bars three times as heavily:
weight = IIf( BarIndex() > LastValue( BarIndex() ) - 200, 3, 1 );
AddNeuralNetworkAmplify( weight, 0 );
TrainMultiInputNeuralNetwork( "MyNetwork" );

The most important practical advice: feed rate of change, not raw price

Scaling cannot rescue an input whose range drifts over time. Raw price is the classic trap: a stock that traded near $0.10 years ago and near $30 today has no stable range, so the scaling computed on old history is wrong for recent bars. The network will happily train and produce a prediction, but it will not be reliable.

The fix is to feed a rate of change rather than the raw level. The percent change of the Close stays within a roughly constant range across the whole history, so it scales cleanly and gives far better, more reliable predictions. The toolbox provides PercentDifference for exactly this; AmiBroker's ROC works too.