Classification of land cover based on hyperspectral data is very challenging because typically tens of classes with uneven priors are involved, the inputs are high dimensional, and there is often scarcity of labeled data. Several researchers have observed that it is often preferable to decompose a multi-class problem into multiple two-class problems, solve each such sub-problem using a suitable binary classifier, and then combine the outputs of this collection of classifiers in a suitable manner to get the answer to the original multiclass problem. This approach is taken by the popular error correcting output codes (ECOC) technique, as well by the binary hierarchical classifier (BHC). Classical techniques for dealing with small sample sizes include regularization of covariance matrices and feature reduction. In this paper we address the twin problems of small sample sizes and multi-class settings by proposing a feature reduction scheme that adaptively adjusts to the amount of labeled data avail able. While exploiting the highly correlated nature of certain adjacent hyperspectral bands. This scheme can be used in conjunction with ECOC and the BHC, as well as other approaches such as round-robin classification that decompose a multi-class problem into a number of two (meta)-class problems. In particular, we develop the best-basis binary hierarchical classifier (BB-BHC) and best basis ECOC (BB-ECOC) families of models that are adapted to ``small sample size'' situations. Currently, there are few studies the compare the efficacy of different approaches to multi-class problems in general sett ings as well as in the specific context of small sample sizes. Our experiments on two sets of remote sensing data show that both BB-BHC and BB-ECOC methods are superior to their non-ad aptive versions when faced with limited data, with the BB-BHC showing s slight edge in terms of classification accuracy as well as interpretability.