Multi-network evolutionary systems and automatic decomposition of complex problems

Multi-network systems, i.e. multiple neural network systems, can often solve complex problems more effectively than their monolithic counterparts. Modular neural networks (MNNs) tackle a complex problem by decomposing it into simpler subproblems and then solving them. Unlike the decomposition in MNNs, a neural network ensemble usually includes redundant component nets and is often inspired by statistical theories. This paper presents different types of problem decompositions and discusses the suitability of various multi-network systems for different decompositions. A classification of various multi-network systems, in the context of problem decomposition, is obtained by exploiting these differences. Then a specific type of problem decomposition, which gives no information about the subproblems and is often ignored in literature, is discussed in detail and a novel MNN architecture for problem decomposition is presented. Finally, a co-evolutionary model is presented, which is used to design and optimize such MNNs with subtask specific modules. The model consists of two populations. The first population consists of a pool of modules and the second population synthesizes complete systems by drawing elements from the pool of modules. Modules represent a part of the solution, which co-operate with each other to form a complete solution. Using two artificial supervised learning tasks, constructed from smaller subtasks, it can be shown that if a particular task decomposition is better than others, in terms of performance on the overall task, it can be evolved using the co-evolutionary model.