Zhiqiang Zhong: Reinforcement Learning based Meta-path Design for Heterogeneous Graph Neural Network

Machine Learning Seminar presentation

Topic: Reinforcement Learning based Meta-path Design for Heterogeneous Graph Neural Network

Speaker: Zhiqiang Zhong, FSTM, University of Luxembourg

Time: Wednesday, 2021.02.10, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Heterogeneous Information Networks (HINs), involving a diversity of node types and relation types, are pervasive in many real-world applications. Recently, increasing attention has been paid to heterogeneous graph representation learning (HGRL) which aims to embed rich structural and semantics information in HIN into low-dimensional node representations. To date, most HGRL models rely on manual customization of meta paths to capture the semantics underlying the given HIN. However, the dependency on the handcrafted meta-paths requires rich domain knowledge which is extremely difficult to obtain for complex and semantic rich HINs. Moreover, strictly defined meta-paths will limit the HGRL’s access to more comprehensive information in HINs.

To fully unleash the power of HGRL, we present a novel framework called RL-HGNN, to design different meta-paths for the nodes in a HIN. Specifically, RL-HGNN models the meta-path design process as a Markov Decision Process and uses a policy network to adaptively design a meta-path for each node to learn its effective representations. The policy network is trained with deep reinforcement learning by exploiting the performance of the model on a downstream task. We further propose an extension, RL-HGNN++, to ameliorate the meta-path design procedure and accelerate the training process. Experimental results demonstrate the effectiveness of RL-HGNN and reveal that it can identify meaningful meta-paths that would have been ignored by human knowledge.

Additional material:

Saurabh Deshpande & Milad Zeraatpisheh: Practical implementation session on Neural Networks

Machine Learning Seminar presentation

Topic: Practical implementation session on Neural Networks

Speaker: Saurabh Deshpande & Milad Zeraatpisheh, FSTM, University of Luxembourg

Time: Wednesday, 2021.02.03, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

We will create our own neural network from scratch, only using the NumPy library. Afterward, the same example will be implemented in the Keras framework. The session will also involve a Bayesian neural network approach, implemented using the TensorFlow probability library.

Additional material:

Shared code: https://colab.research.google.com/drive/1aTf490wwL7r8_eEjcW4Sw9rgPg6ntVvf#scrollTo=B1OA9W_KnmUo

Arif Sinan Uslu: The effect of self-paced neurofeedback on EEG learning: An experimental setup

Machine Learning Seminar presentation

Topic: The effect of self-paced neurofeedback on EEG learning: An experimental setup

Speaker: Arif Sinan Uslu, FHSE, University of Luxembourg

Time: Wednesday, 2021.01.27, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Neurofeedback refers to the regulation of brain activity via a brain-computer interface. In a feedback loop, the brain activity is recorded, analyzed, and fed back to the user. Iterative and dynamic feedback enables the user to learn whether or not the recorded brain activity meets a pre-specified threshold. Research in the field of cognitive electrophysiology suggests that the regulation of brain rhythms is associated with changes in cognitive functions [1,2]. However, these associations frequently apply to subgroup of participants who were able to regulate their brain activity [2,3,4]. Up until now, most studies on EEG neurofeedback applied a block design during training during which participants receive the same timed neurofeedback training blocks (e.g. 5 minutes) interspersed by rest periods. To my knowledge, there has been no study investigating the effect of self-paced neurofeedback on learning outcome although the factor of training frequency and duration has been discussed as an indicator of training success. In the current study, I will investigate how self-paced neurofeedback influences the activity of alpha frequency compared to classical externally-paced neurofeedback and a control group receiving sham-neurofeedback.

In my presentation, I will give a brief theoretical introduction to the matter and focus later on the technical implementation of my experimental setup and the type of data that is being recorded. Although a data-driven methodology is not implemented for this experiment, I would very much like to take this opportunity to discuss possibilities to include data-driven methods that could be applied to the data from this experiment or a modified version of this experiment.

Additional material:

[1] Escolano, C., Navarro-Gil, M., Garcia-Campayo, J. et al. The Effects of a Single Session of Upper Alpha Neurofeedback for Cognitive Enhancement: A Sham-Controlled Study. Appl Psychophysiol Biofeedback 39, 227–236 (2014). https://doi.org/10.1007/s10484-014-9262-9
[2] Hanslmayr, S., Sauseng, P., Doppelmayr, M., Schabus, M., & Klimesch, W. (2005). Increasing individual upper alpha power by neurofeedback improves cognitive performance in human subjects. Applied psychophysiology and biofeedback, 30(1), 1–10. https://doi.org/10.1007/s10484-005-2169-8
[3] Nan, W., Rodrigues, J. P., Ma, J., Qu, X., Wan, F., Mak, P. I., Mak, P. U., Vai, M. I., & Rosa, A. (2012). Individual alpha neurofeedback training effect on short term memory. International journal of psychophysiology : official journal of the International Organization of Psychophysiology, 86(1), 83–87. https://doi.org/10.1016/j.ijpsycho.2012.07.182
[4] Kober, S. E., Witte, M., Ninaus, M., Neuper, C., & Wood, G. (2013). Learning to modulate one’s own brain activity: the effect of spontaneous mental strategies. Frontiers in human neuroscience, 7, 695. https://doi.org/10.3389/fnhum.2013.00695 (edited) 

Milad Zeraatpisheh: Bayesian neural networks and MC Dropout; ways to measure uncertainty in Deep learning

Machine Learning Seminar presentation

Topic: Bayesian neural networks and MC Dropout; ways to measure uncertainty in Deep learning

Speaker: Milad Zeraatpisheh, FSTM, University of Luxembourg

Time: Wednesday 2021.01.20, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Deep learning methods represent the state-of-the-art for numerous applications such as facial recognition systems, supercomputing, and speech recognition. Conventional Neural networks generate point estimates of deep neural network parameters and therefore, make predictions that can be overconfident since they do not account well for uncertainty in model parameters.

In this presentation, we will take a closer look at the Bayesian Neural network as a way to measure this uncertainty. First, Bayesian inference on Neural network weights will be discussed. Afterward, Monte Carlo Dropout, proposed by Gal & Ghahramani (2016), as another way to tackle uncertainty in deep learning will be explained.

Additional material:

 

Lars Beex: Unsupervised learning to select modes for reduced-order models of hyperelastoplasticity: application to RVEs

Machine Learning Seminar presentation

Topic: A Physics-guided Machine Learning Model Based on Peridynamics

Speaker: Dr Lars Beex, FSTM, University of Luxembourg

Authors: S. Vijayaraghavan (1; 2), L.A.A.Beex (1), L.Noels (2), and S.P.A.Bordas (1)

1) University of Luxembourg, Ave. de la Fonte 6, L-4364 Belval, Luxembourg.

2) University of Liege, CM3 B52, Alle de la découverte 9, B4000 Liege, Belgium

Time: Wednesday 2021.01.13, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Many model order reduction approaches use solutions of a few ‘offline’ training simulations to reduce the number of degrees of freedom of the many ‘online’ simulations of interest. In proper orthogonal decomposition, singular value decomposition is applied to a matrix with the training solutions in order to capture the most essential characteristics in the first few modes – which are used as global interpolation bases.

Proper orthogonal decomposition has proven itself as an accurate reduced-order model approach for elliptical partial differential equations. In the field of solid mechanics, this means that it is accurate for (hyper)elastic material models, but not for (hyper)elastoplasticity. Based on the study of [1], the current contribution investigates how clustering of the training solutions and extracting global modes from each cluster can improve the accuracy of proper orthogonal decomposition for hyperelastoplasticity. Both centroid-based clustering (i.e. k-means clustering) and connectivity-based clustering (based on Chinese whispers) are investigated.

The approach is applied to a hyperelastoplastic representative volume element exposed to monotonic loading, quasi-monotonic loading, and quasi-random loading. In case of monotonic and quasi-monotonic loading, the components of the macroscopic deformation tensor are the variables to which clustering is applied. In case of quasi-random loading, however, not only the components of the macroscopic deformation tensor and the incremental changes of these components are the variables to which clustering is applied, but also all history variables of all integration points.

Additional material:

[1] David Amsallem, Matthew J. Zahr and Charbel Farhat. Nonlinear model order reduction based on local reduced-order bases. Int. J. Numer. Meth. Engng. VOL 92 IS-10 SN-0029-5981

Erkan Oterkus: A Physics-guided Machine Learning Model Based on Peridynamics

Machine Learning Seminar presentation

Topic: A Physics-guided Machine Learning Model Based on Peridynamics

Speaker: Pofessor Erkan Oterkus, PeriDynamics Research Center, University of Strathclyde, Glasgow, UK

Time: Wednesday 2020.12.16, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

With the rapid growth of available data and computing resources, using data-driven models is a potential approach in many scientific disciplines and engineering. However, for complex physical phenomena that have limited data, the data-driven models are lacking robustness and fail to provide good predictions. Theory-guided data science is the recent technology that can take advantage of both physics-driven and data-driven models. In this webinar, a new physics-guided machine learning model based on peridynamics will be presented. Peridynamics is a suitable approach for predicting progressive damages because the theory uses integro-differential equations instead of partial differential equations. Several numerical examples will be shown to demonstrate the capability of the methodology.

Additional material:

 

Eleni Koronaki: From partial data to out-of-sample parameter and observation estimation with Diffusion Maps and Geometric Harmonics

Machine Learning Seminar presentation

Topic: From partial data to out-of-sample parameter and observation estimation with Diffusion Maps and Geometric Harmonics

Speaker: Dr. Eleni Koronaki, University of Luxembourg, Department of Computational Science

Time: Wednesday 2020.12.09, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

A data-driven framework is presented, that enables the estimation of quantities, either observations or parameters, given enough partial data. Here the data are high-dimensional vectors containing the outputs of a detailed CFD model of the process, i.e. the values of velocity, pressure, temperature and species mass fractions at each point in the discretization. The goal is to predict the outputs of new inputs, here process parameters and also the inputs that correspond to new outputs.

Finally, along the lines of a nonlinear observer, part of the outputs are predicted given only a limited number of partial observations, i.e. not the entire output vector but rather a handful of independent values. The proposed workflow begins by determining the intrinsic reduced description of the available data with Diffusion Maps, a data-driven nonlinear manifold learning technique that can be thought of as the nonlinear counterpart of Principal Components Analysis. This low-dimensional description of the high dimensional ambient space that contains the data, is then leveraged for efficient interpolation and regression with a special implementation of Geometric Harmonics.

Additional material:

 

Vivek OOmmen: The effectiveness of PINNs in solving inverse heat transfer problems

Machine Learning Seminar presentation

Topic: The effectiveness of PINNs in solving inverse heat transfer problems

Speakers: Vivek Oommen & Prof. Balaji Srinivasan, University of Luxembourg, Department of Computational Science

Time: Wednesday 2020.12.02, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

In this presentation, I would like to demonstrate the effectiveness of PINNs in solving several inverse heat transfer problems. To justify its effectiveness, the time taken by PINNs to solve the entire problem is compared with the time taken by COMSOL Multiphysics Software (FEM approach) to solve the forward problem once. Steady and unsteady problems governed by both linear and nonlinear PDEs have been chosen as the test cases, to check if PINNs can handle a variety of problems often encountered in the field of heat transfer. I will conclude after mentioning a specific type of PINN architecture that works very well for PDEs that are highly nonlinear.

Additional material:

[1] “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations” (M. Raissi, P. Perdikaris and G.E. Karniadakis), (Journal of Computational Physics)
[2] “Estimation of the non-linear diffusion coefficient with Markov Chain Monte Carlo method based on the integral information” (Zbigniew Bulinski and Helcio R.B. Orlande) (International Journal of Numerical Methods for Heat & Fluid Flow)
[3] “A Bayesian approach for the simultaneous estimation of surface heat transfer coefficient and thermal conductivity from steady state experiments on fins” (N. Gnanasekaran and C. Balaji), (International Journal of Heat and Mass Transfer)
[4] “Application of genetic algorithm for unknown parameter estimations in cylindrical fin” (Ranjan Das), (Applied Soft Computing)
[5] “Application of Adomian decomposition method and inverse solution for a fin with variable thermal conductivity and heat generation” (Rohit K. Singla and Ranjan Das), (International Journal of Heat and Mass Transfer)
[6] “Prediction of Heat Generation in a Porous Fin from Surface Temperature” (Ranjan Das and Balaram Kundu), (International Journal of Heat and Mass Transfer)

 

Anina Šarkić: Machine Learning in Wind Engineering

Machine Learning Seminar presentation

Topic: Machine Learning in Wind Engineering

Speaker: Dr. Anina Šarkić, University of Luxembourg, Department of Computational Science

Time: Wednesday 2020.11.25, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

With the development of construction technology and materials, more light and super-flexible structures have been built all over the world. As a consequence, they are becoming very sensitive to the wind loads and, in addition, more complex wind flow patterns are developed. In design phase, this more detailed wind loads and flow information comes from wind tunnel tests. However, they are very costly when several geometrical scenarios are to be analyzed. In addition, wind tunnels also do not naturally allow the monitoring of important quantities of interest over large control volumes. Another approach that can overcome some of the drawbacks of wind tunnel methodology relays on computational fluid dynamics, yet cannot be used in isolation from wind tunnels.

Therefore, this presentation explores other effective ways that may go beyond existing methodologies that rely on machine learning. In particular, prediction of wind loads on the high-rise building using backpropagation neural network combined with POD will be shown.

Additional material:

[1] Dongmei, H., Shiqing, H., Xuhui, H., Xue, Z., (2017), Prediction of the wind loads on high-rise buildings using BP neural network combined with POD, Journal of wind engineering and industrial aerodynamics, 170, 1-17. (https://www.scribd.com/document/396693163/Dongmei-2017)

[2] Tian, J., Gurley, K., R., Diaz., M., T., Fernandez-Caban, P., L., Masters, F., J., Fang, R., (2020), Low-rise gable roof buildings pressure prediction using deep neural networks, Journal of wind engineering and industrial aerodynamics, 196 (https://doi.org/10.1016/j.jweia.2019.104026)

[3] Bernardini, E., Spence, S., M., J., Wie, D., Kareem, A., (2015), Aerodinamic shape optimization of civil structures: A CFD-enabled Kringing-based approach, Journal of wind engineering and industrial aerodynamics, 144, 154-164. (https://doi.org/10.1016/j.jweia.2015.03.011)

 

Diego Kozlowski: Machine Learning on graphs

Machine Learning Seminar presentation

Topic: Machine Learning on graphs

Speaker: Diego Kozlowski, University of Luxembourg, Department of Computational Science

Time: Wednesday 2020.11.18, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Graphs are a ubiquitous data structure that can be exploited in many different problems. In tasks where observations are not independently drawn from the data generating process, but their codependencies add valuable information, a network analysis might be useful for modelling those relations.

In this seminar, we will discuss Graph Neural Networks, the deep learning approach for dealing with networks.

 

Additional material:

[1] Hamilton, W. L. (2020). Graph representation learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 14(3), 1-159. (https://www.cs.mcgill.ca/~wlh/grl_book/files/GRL_Book.pdf)

[2] Bacciu, D., Errica, F., Micheli, A., & Podda, M. (2020). A gentle introduction to deep learning for graphs. Neural Networks. (https://arxiv.org/abs/1912.12693)

[3] Hamilton, W. L., Ying, R., & Leskovec, J. (2017). Representation learning on graphs: Methods and applications. arXiv preprint (https://arxiv.org/abs/1709.05584)

[4] Bronstein, M. M., Bruna, J., LeCun, Y., Szlam, A., & Vandergheynst, P. (2017). Geometric deep learning: going beyond Euclidean data. IEEE Signal Processing Magazine, 34(4), 18-42. (https://arxiv.org/abs/1611.08097)