## Workshop "Current Problems of Neuroinformatics"

### Neuroinformatics - 2017

Tuesday, October 3 11:00 – 13:00

Lecture-hall Алексеевский зал

Chair: Prof. MAKARENKO NIKOLAY

*Scientific Research Institute for System Analysis, Moscow*

**N-vicinity method and calculation of free energy**

Methods of statistical physics are used not in physics only but also in the combinatorial optimization, the theory of neural networks, machine learning and computer processing of images. In all these problems calculation of free energy is one of the key problems because it is necessary to calculate many important macroscopic characteristics of a system such as magnetization, heat capacity, susceptibility and so on. To calculate free energy we developed an n-vicinity method. Our approach does not depend on specifics of the connection matrix and allows us to introduce an external magnetic field. When applied to the Ising model on a $D$-dimensional cubic lattices, it gave a simple analytic solution for critical values of the inverse temperature that describes well computer simulations for a number of dimensions D = 3, 4, 5, 6 and 7. Here we explain briefly the statistical physics approach and discuss the n-vicinity method.

2. TEREKHOV S.A.

**Tensor decomposition of multidimensional data in statistical modeling**

Statistical learning methods are considered in low-rank tensor decompositions of multidimensional counts data with missing values. Special case of right-censored data is discussed in detail. Numerous applications of tensor factorizations in logistics, resource and capacity planning and optimizations, as well as recommender systems are presented.

Wednesday, October 4 11:00 – 13:00

Lecture-hall Алексеевский зал

Chair: Prof. TEREKHOV SERGE

*The Moscow Institute of Physics and Technology (State University)*

**Deep learning. 10 years later**

About 10 years ago several research groups, namely G. Hinton, Y. Bengio and Y. LeCun with co-authors revived interest in neural networks, starting the deep learning revolution, and refreshing the field of machine intelligence. Deep neural networks with hundreds of layers and billions of weights replaced modestly sized 2--3-layered networks of 1990s. Parallel deep learning algorithms making use of powerful GPU accelerators managed to solve long-known tough problems from machine vision to machine translation. Machine intelligence came out of labs to real-life applications such as smart agents, self-driving cars, drones and robots, and is developing at an increasingly high pace. This survey aims to present deep learning revolution in its historical retrospective.

4. G. OSOSKOV, P. GONCHAROV, A.TSYTRINOV

1

*Joint Institute for Nuclear Research*

2

*Sukhoi State Technical University of Gomel*

**Deep neural networks for image classification**

Our report intends to demonstrate the advantages of deep learning approaches over ordinary shallow neural network on their comparative applications to image classifying. A deep autoassociative neural network is used as a standalone autoencoder for prior extracting the most informative features of input data for neural networks to be compared further as classifiers. The main efforts to deal with deep learning networks are spent for a quite painstaking work of optimizing structures of those networks and their components, as activation functions, weights, as well as the procedures of minimizing their loss function to improve their performances and speed up their learning time. It is also shown deep autoencoders develop the remarkable ability for denoising images after being specially trained. Convolutional Neural Networks are also used to solve a quite actual problem of protein genetics on the example of the durum wheat classification. Results of our comparative study demonstrate the undoubted advantage of deep networks, as well as denoising power of autoencoders. In our work we use both GPU and cloud services to speed up our calculations.

Thursday, October 5 11:00 – 12:45

Lecture-hall Алексеевский зал

Chair: Prof. TIUMENTSEV YURY

*Siberian Federal University, Krasnoyarsk*

**Application of artificial intelligence methods and neural network technology for adaptive numerical methods of global optimization**

We present variants of efficient computer methods of global minimum search for black box objective functions, which are based on artificial intelligence methods and neural network technologies. Modifications of the Particle Swarm Optimization (PSO) method reinforcing swarm intelligence with neuro--fuzzy control of particle movement are considered. The neural network approximation of inverse coordinate mappings method is described. The results of global minimum search experiments are provided for test objective functions of 2 (for clarity), 50, 100, 500 variables.

6. SHIBZUKHOV Z.M.

**Training of robust neural networks**

The lecture is devoted to the problems of robust machine learning of parametric models and neural networks (NN) in the context of outliers. The aspects of the problem of robust machine learning and the method of iterative re-weighting are discussed. In order to search for optimal weights of regression parametric models and NN we consider more advanced robust machine learning procedures that based on the minimization of the average aggregating function from losses and squared errors, in particular. It is shown that in this case it is possible to apply a weighted method of learning such as back propagation for the training of robust NN based on the application of averaging aggregating functions, which in a certain sense approximate the median and quantiles.

Friday, October 6 11:00 – 13:00

Lecture-hall Алексеевский зал

Chair: Prof. DOLENKO SERGEY

*Moscow Aviation Institute (National Research University)*

**Neural network identification of characteristics of nonlinear controlled dynamical systems**

In the process of creation and operation of aircrafts of various types, the solution of such classes of problems as the analysis of the behavior of dynamical systems, the synthesis of control algorithms for them, the identification of their unknown or inaccurately known characteristics occupies a significant place. A critical role in solving these problems belongs to mathematical and computer models of dynamical systems. An approach to the formation of such models is considered, based on combining theoretical knowledge about the modeling object with methods and tools of neural network modeling.

8. VASILIEV A.N., TARKOV M.S.

*Peter the Great St. Petersburg Polytechnic University*

**Unified process for construction of real objects' mathematical models based on approximate neural solutions of differential equations with other heterogeneous data**

The lecture describes the basics of the methodology developed by the authors to construct approximate neural network solutions of ordinary and partial differential equations. The methodology is illustrated with examples of tasks: for compound domains (with a different type of equations in the subdomains), for a variable boundary (caused by phase transition), for a selected boundary, with some heterogeneous information, which includes differential equations, initial and boundary conditions, experimental and other data. It is shown that these different tasks are solved uniformly with the application of general principles. A new approach to the construction of multilayer neural network for solving partial differential equations is given; this approach allows obtaining arbitrarily accurate solutions without time-consuming learning procedure.

Friday, October 6 14:00 – 14:45

Lecture-hall Алексеевский зал

Chair: Prof. DOLENKO SERGEY

*The Central Astronomical Observatory of the Russian Academy of Sciences at Pulkovo, Saint-Petersburg*

**How to measure a point cloud?**

In the problems of pattern recognition, you often have to deal with a cloud of points. Such a discrete set can be vectors from the feature space, the selection of points from the surface of a certain manifold, the persistence diagram, as a result of topological filtering of the random field, the phase points of the reconstruction of the dynamic system obtained from the observed time series by the Takens algorithm. Training in these cases is not an easy task. In the general case, the cloud is neither a vector space nor a manifold. Necessary procedures, such as methods for measuring the distances between two clouds and the technique of their averaging in such a set, are absent. His suitable mathematical image is the Alexandrov space. Transport metrics such as Wasserstein distances are computationally complex. The transition to probability densities does not alleviate the problem - one has to deal with distances, such as the Kulbak-Leibler or Bregman divergence. The lecture tells how to equip the cloud with a suitable Hilbert space, with familiar and convenient ways of obtaining measures of proximity and averaging. One of the most beautiful techniques is the transformation of probability densities into points of a single hypersphere with a Riemannian metric. This method is similar to the transition from probabilities to their amplitudes, the wave function of quantum mechanics. Formalism is demonstrated on practical examples with the recognition of textures and the diagnosis of random fields.