Multiresolution feature
extraction for pairwise classification of hyperspectral data
S. Kumar, J. Ghosh and M. M. Crawford
Learning a large number of simple local concepts is both faster and
easier than learning a single global concept. Inspired by this
principle of divide and conquer, a number of modular learning
approaches have been proposed by the computational intelligence
community. In modular learning, the classification/regression/clustering
problem is first decomposed into a number of simpler subproblems, a
module is learned for each of these subproblems, and finally their
results are integrated by a suitable combining method. Mixtures of
experts and clustering are two of the techniques that are
describable in this paradigm. In this paper we present a broad
framework for Generalized Associative Modular Learning Systems
(GAMLS). Modularity is introduced through soft association of each
training pattern with every module. The coupled problems of learning
the module parameters and learning associations are solved
iteratively using deterministic annealing. Starting at a high
temperature with only one module, GAMLS framework automatically
evolves the required number of modules through a systematic growing
and pruning technique. Each phase begins by splitting every module in
the previous phase into two, updating these new modules and then
pruning and merging any redundant modules. A phase transition is
induced by temperature decay. A number of existing modular learning
problems, both unsupervised (clustering, mixture model density,
mixture of principal components) and supervised (mixture of experts,
radial basis function networks), can be effectively tackled in
GAMLS. Case studies for clustering and regression using mixture of
experts are provided for a number of datasets showing the efficacy of
the GAMLS framework in evolving the right number of modules, inducing
interpretable localizations among modules and robustness of the
solution obtained. More importantly, this framework provides a
unifying view for understanding and characterizing modular learning
methods.