2021 IEEE International Conference on Image Processing
19-22 September 2021 • Anchorage, Alaska, USA
Imaging Without Borders
| Paper ID | MLR-APPL-IVSMR-1.1 | ||
| Paper Title | A SIMPLE, EFFECTIVE WAY TO IMPROVE NEURAL NET CLASSIFICATION: ENSEMBLING UNIT ACTIVATIONS WITH A SPARSE OBLIQUE DECISION TREE | ||
| Authors | Arman Zharmagambetov, Miguel Á. Carreira-Perpiñán, University of California, Merced, United States | ||
| Session | MLR-APPL-IVSMR-1: Machine learning for image and video sensing, modeling and representation 1 | ||
| Location | Area C | ||
| Session Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation Time: | Tuesday, 21 September, 13:30 - 15:00 | ||
| Presentation | Poster | ||
| Topic | Applications of Machine Learning: Machine learning for image & video sensing, modeling, and representation | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | We propose a new type of ensemble method that is specially designed for neural nets, and which produces surprising improvements in accuracy at a very small cost, without requiring to train a new neural net. The idea is to concatenate the output activations of internal layers of the neural net into an ``ensemble feature vector'', and train on this a decision tree to predict the class labels while also doing feature selection. For this to succeed we rely on a recently proposed algorithm to train decision trees - Tree Alternating Optimization. This simple procedure consistently improves over simply ensembling the nets in the traditional way, achieving relative error decreases of well over 10% of the original nets on the well known image classification benchmarks. As a subproduct, we also can obtain an architecture consisting of a neural net feature extraction followed by a tree classifier that is faster and more compact than the original net. | ||