Fateme Darlik: Using Physics-Informed constrained neural network to reconstruct the motion of the particles for time more than the training time

Fateme Darlik: Using Physics-Informed constrained neural network to reconstruct the motion of the particles for time more than the training time

Machine Learning Seminar presentation

Topic: Using Physics-Informed constrained neural network to reconstruct the motion of the particles for time more than the training time.

Speaker: Fateme Darlik, Faculty of Science Technology and Medicine, University of Luxembourg.

Time: Wednesday, 2022.06.22, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Physics-informed neural networks (PINNs) have been studied and used widely in the field of computational mechanics. In this study, we proposed a new approach to merge PINNs with (geometry-based) constrained neural networks. Using this method, we reconstruct the velocity fields of particles moving in the fixed bed. Following that, the trained neural network is used to predict the motion of particles for a period of time more than the training time, and the results are compared with the simulation data in terms of accuracy and CPU time.

Additional material

Video recording: https://youtu.be/yQXZ2tNT-Mw

Aishwarya Krishnan: Design of a chat bot for service-now knowledge base

Aishwarya Krishnan: Design of a chat bot for service-now knowledge base

Machine Learning Seminar presentation

Topic: Design of a chat bot for service-now knowledge base.

Speaker: Aishwarya Krishnan, Faculty of Science Technology and Medicine, University of Luxembourg

Time: Wednesday, 2022.06.01, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Service now is an important tool in an industry that helps internal employees to report and solve their issues. In an organization that receives an average of 7000+ IT related incidents in six months, human assisted resolving of issues is cumbersome. Therefore, my thesis focuses on designing an artificial intelligence assisted chat-bot that uses machine learning algorithms to bring up a precise solution that could bring an automated response to the user. In this talk I will be discussing about the challenges that we face in an industrial setting, the roughness of data, prepossessing them and finally retrieving the relevant information from the data. I will be discussing about the libraries that was used to simplify the text data and the type conversions involved. I propose to discuss in this presentation about the input given to the Bert model and the working of the transformer model itself.

Additional material:

Video recording: https://youtu.be/1WqA_KL7lPM

Lorella Viola: The problem with GPT for the Humanities (and for humanity)

Lorella Viola: The problem with GPT for the Humanities (and for humanity)

Machine Learning Seminar presentation

Topic: The problem with GPT for the Humanities (and for humanity).

Speaker: Lorella Viola, Luxembourg Centre for Contemporary and Digital History (C2DH ), University of Luxembourg

Time: Wednesday, 2022.05.11, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

GPT models (Generative Pre-trained Transformer) have increasingly become a popular choice by researchers and practitioners. Their success is mostly due to the technology’s ability to move beyond single word predictions. Indeed, unlike in traditional neural network language models, GPT generates text by looking at the entirety of the input text. Thus, rather than determining relevance sequentially by looking at the most recent segment of input, GPT models determine each word’s relevance selectively. However, if on the one hand this ability allows the machine to ‘learn’ faster, the datasets used for training have to be fed as one single document, meaning that all metadata information is inevitably lost (e.g., date, authors, original source). Moreover, as GPT models are trained on crawled, English web material from 2016, these models are not only ignorant of the world prior to this date, but they also express the language as used exclusively by English-speaking users (mostly white, young males). They also expect data pristine in quality, in the sense that these models have been trained on digitally-born material which do not present the typical problems of digitized, historical content (e.g., OCR mistakes, unusual fonts). Although a powerful technology, these issues seriously hinder its application for humanistic enquiry, particularly historical. In this presentation, I discuss these and other problematic aspects of GPT and I present the specific challenges I encountered while working on a historical archive of Italian American immigrant newspapers.

Additional material:

Video recording: https://youtu.be/J1zkS1mKGlE

Aubin Geoffre: Gaussian Process Regression with anisotropic kernel for supervised feature selection.

Aubin Geoffre: Gaussian Process Regression with anisotropic kernel for supervised feature selection.

Machine Learning Seminar presentation

Topic: Gaussian Process Regression with anisotropic kernel for supervised feature selection.

Speaker: Aubin Geoffre, École Mines de Saint-Étienne

Time: Wednesday, 2022.05.04, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Studying flows in random porous media leads to consider a permeability tensor which directly depends on the pore geometry. The latter can be characterised through the computation of various morphological parameters: Delaunay triangulation characteristics, nearest neighbour distance,… The natural question is: which morphological parameters provide the best statistical description of permeability? This question can be difficult to answer for several reasons: non-linear correlation between input parameters, non-linear correlation between inputs and outputs, small dataset, variability,…
A method of feature selection based on Gaussian Process Regression has been proposed. It can be applied to a wide range of applications where the parameters that best explain a given output are sought among a set of correlated features. The method uses anisotropic kernel that associates a hyperparameter to each feature. These hyperparameters can be interpreted as correlation lengths providing an estimation of the weight of each feature w.r.t the output.

Additional material:

Video recording: https://youtu.be/S7_RcE2WmGk

Presentation: https://legato-team.eu/wp-content/uploads/2022/05/seminar_legato.pdf

Vu Chau: Non-parametric data-driven constitutive modelling using artificial neural networks.

Vu Chau: Non-parametric data-driven constitutive modelling using artificial neural networks.

Machine Learning Seminar presentation

Topic: Non-parametric data-driven constitutive modelling using artificial neural networks.

Speaker: Vu Chau, Department of Engineering, FSTM, University of Luxembourg

Time: Wednesday, 2022.04.13, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

This seminar talk addresses certain challenges associated with data-driven modelling of advanced materials — with special interest in the non-linear deformation response of rubber-like materials, soft polymers or biological tissue. The underlying (isotropic) hyperelastic deformation problem is formulated in the principal space, using principal stretches and principal stresses. The sought data-driven constitutive relation is expressed in terms of these principal quantities and to be captured by a non-parametric representation using a trained artificial neural network (ANN).
The presentation investigates certain physics-motivated consistency requirements (e.g. limit behaviour, monotonicity) for the ANN-based prediction of principal stresses for given principal stretches, and discusses the implications on the architecture of such constitutive ANNs. The neural network is exemplarily constructed, trained and tested using PyTorch.
The computational embedding of the data-driven material descriptor is demonstrated for the open-source finite element framework FEniCS which builds on the symbolic representation of the constitutive ANN operator in the Unified Form Language (UFL). We discuss the performance of the overall formulation within the non-linear solution process and will explain some future directions of research.

Additional material:

Video recording: https://youtu.be/Vn0fLGHAkbk

Alban Odot: Deformation approximation : Improve your Artificial Neural Network training using the Finite Element Method formulation. A case for static deformations.

Alban Odot: Deformation approximation : Improve your Artificial Neural Network training using the Finite Element Method formulation. A case for static deformations.

Machine Learning Seminar presentation

Topic: Deformation approximation: Improve your Artificial Neural Network training using the Finite Element Method formulation. A case for static deformations.

Speaker: Alban Odot, Ph.D. student at INRIA Strasbourg (MIMESIS-Inria)

Time: Wednesday, 2022.03.16, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Recently, with the increase in GPU computational power, deep learning started to revolutionise several fields, in particular computer vision, language processing, and image processing. Deep learning engineering can be split into three main categories: Dataset, Network Architecture and Learning policy. Setting the network architecture to its simplest form, we will modify the dataset and learning policy formulation using the Finite Element Method to improve the training.
The Finite Element Method is often used as the numerical method of reference for solving the PDE associated with non-linear object deformations. In order to solve the resulting energy minimisation equations, root-finding algorithms such as the Newton-Raphson method are used. During its iterative process, the Newton-Raphson reveals important information about the state of the system which can be used in both the dataset formulation and the training policy.

Additional material:

Video recording: https://youtu.be/FNIX0XCix8c

Onkar Jadhav: Parametric model order reduction with an adaptive greedy sampling approach based on surrogate modeling. An application of the pMOR in financial risk analysis.

Onkar Jadhav: Parametric model order reduction with an adaptive greedy sampling approach based on surrogate modeling. An application of the pMOR in financial risk analysis.

Machine Learning Seminar presentation

Topic: Parametric model order reduction with an adaptive greedy sampling approach based on surrogate modeling. An application of the pMOR in financial risk analysis.

Speaker: Onkar Jadhav, Research Associate, Interdisciplinary Centre for Security, Reliability and Trust (SnT), University of Luxembourg.

Time: Wednesday, 2022.03.02, 15:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

The numerical simulation of physical systems typically involves the solution of large-scale systems of equations resulting from the discretization of PDEs. Model order reduction techniques are very advantageous to overcome this computational hurdle. Based on the proper orthogonal decomposition approach, the talk will present a model order reduction approach for parametric high dimensional convection-diffusion-reaction partial differential equations. The proper orthogonal decomposition requires solving the high dimensional model for some training parameters to obtain the reduced basis. In this work, the training parameters are chosen based on greedy sampling approaches. We propose an adaptive greedy sampling approach that utilizes an optimized search based on surrogate modeling for the selection of the training parameter set. The work also presents an elegant approach for monitoring the convergence of the developed greedy sampling approach along with its error and sensitivity analysis.
The developed techniques are analyzed, implemented, and tested on industrial data of a floater with caps and floors under the one-factor Hull-White model. The results illustrate that the model order reduction approach provides a significant speedup with excellent accuracy for short-rate models.

Additional material:

Video recording: https://youtu.be/rJAUDG6Y0Pc

Lester Mackey: Kernel Thinning and Stein Thinning

Lester Mackey: Kernel Thinning and Stein Thinning

Machine Learning Seminar presentation

Topic: Kernel Thinning and Stein Thinning

Speaker: Lester Mackey, a researcher at Microsoft Research New England, and an adjunct professor at Stanford University.

Time: Wednesday, 2022.02.23, 15:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

This talk will introduce two new tools for summarizing a probability distribution more effectively than independent sampling or standard Markov chain Monte Carlo thinning:

1. Given an initial n point summary (for example, from independent sampling or a Markov chain), kernel thinning finds a subset of only square-root n points with comparable worst-case integration error across a reproducing kernel Hilbert space.

2. If the initial summary suffers from biases due to off-target sampling, tempering, or burn-in, Stein thinning simultaneously compresses the summary and improves the accuracy by correcting for these biases.

These tools are especially well-suited for tasks that incur substantial downstream computation costs per summary point like organ and tissue modeling in which each simulation consumes 1000s of CPU hours.

Additional material:

Video recording: https://youtu.be/91p4Octzc6E

Ola Rønning: ELBO-within-Stein: General and integrated Stein Variational Inference (Part 3 of 3)

Ola Rønning: ELBO-within-Stein: General and integrated Stein Variational Inference (Part 3 of 3)

Machine Learning Seminar presentation

Topic: ELBO-within-Stein: General and integrated Stein Variational Inference (Part 3 of 3)

Speaker: Ola RønningPhD student in the Probabilistic programming group of Prof. Thomas Hamelryck, Dpt. of Computer Science, University of Copenhagen

Time: Wednesday, 2022.02.16, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Bayesian inference provides a unified framework for quantifying uncertainty in probabilistic models with latent variables. However, exact inference algorithms generally scale poorly with the dimensionality of the model and the size of the data. To overcome the issue of scaling, the ML community has turned to approximate inference. For the Big Data case, the most prominent method is Variational inference (VI), which uses a simpler parametric model to approximate the target distribution of the latent variables. In recent years, Stein’s method has caught the attention of the ML community as a way to formulate new schemes for performing variational inference. Stein’s method provides a fundamental technique for approximating and bounding distances between probability distributions. The kernel Stein discrepancy underlies Stein Variational Gradient Descent (SVGD) which works by iteratively transporting particles sampled from a simple distribution to the target distribution. We introduce the ELBO-within-Stein algorithm that combines SVGD and VI to alleviate issues due to high-dimensional models and large data sets. The ELBO-within-Stein algorithm is available in our computational framework EinStein distributed with the deep probabilistic programming language NumPyro. We will draw upon our framework to illustrate key concepts with examples. EinStein is currently freely available on GitHub and will be available in NumPyro from the next release. The framework is an efficient inference tool for practitioners and a flexible and unified codebase for researchers.

Additional material:

Video recording: https://youtu.be/vsI7J5pgTv0

Google Colab: https://colab.research.google.com/drive/17vAcyDXbcuDRj8IUym8zdP0OxN5kpLmD?usp=sharing

Christian Thygesen: Efficient generative modelling of protein structure fragments using a Deep Markov Model (Part 2 of 3)

Christian Thygesen:  Efficient generative modelling of protein structure fragments using a Deep Markov Model (Part 2 of 3)

Machine Learning Seminar presentation

Topic: Efficient generative modelling of protein structure fragments using a Deep Markov Model (Part 2 of 3)

Speaker: Christian ThygesenIndustrial PhD student with Evaxion / University of Copenhagen

Time: Wednesday, 2022.02.09, 10:00 CET

How to join: Please contact Jakub Lengiewicz

Abstract:

Fragment libraries are often used in protein structure prediction, simulation and design as a means to significantly reduce the vast conformational search space. Current state-of-the-art methods for fragment library generation do not properly account for aleatory and epistemic uncertainty, respectively due to the dynamic nature of proteins and experimental errors in protein structures. Additionally, they typically rely on information that is not generally or readily available, such as homologous sequences, related protein structures and other complementary information.
To address these issues, we developed BIFROST, a novel take on the fragment library problem based on a Deep Markov Model architecture combined with directional statistics for angular degrees of freedom, implemented in the deep probabilistic programming language Pyro. BIFROST is a probabilistic, generative model of the protein backbone dihedral angles conditioned solely on the amino acid sequence. BIFROST generates fragment libraries with a quality on par with current state-of-the-art methods at a fraction of the run-time, while requiring considerably less information and allowing efficient evaluation of probabilities.

Additional material:

Video recording: https://youtu.be/hydwEaq_Uuo