Advanced Non-Homogeneous Model Order Reduction for Neural Network Compression and Complexity Reduction of AI Systems

Description

Modern AI systems have grown to hundreds of billions of parameters, creating urgent memory, latency, and energy bottlenecks, especially for edge and real-time deployment. Model Order Reduction (MOR), a mature discipline from control theory, has recently been adapted for neural network compression through layer-wise hybrid frameworks (e.g., HMORF) that combine balanced truncation, POD, and tensor decomposition with provable linearised error bounds. However, all existing MOR-for-AI approaches are homogeneous: they apply the same reduction technique, uniform reduction order, or fixed error tolerance across every layer. This ignores the well-documented fact that layer sensitivity is highly non-uniform, some layers tolerate drastic reduction with negligible accuracy loss while others are critically sensitive, and uniform allocation is now recognised as a fundamental limitation ("uniformity trap") of one-size-fits-all compression.