Portfolio item number 1
Short description of portfolio item number 1
Posts by Collection
portfolio
Portfolio item number 2
Short description of portfolio item number 2
publications
Published in , 1900
DeepONet Based Preconditioning Strategies for Solving Parametric Linear Systems of Equations.
A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multi-level feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behaviour in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.
Published in , 1900
Parallel Trust-Region Approaches in Neural Network Training: Beyond Traditional Methods.
We propose to train neural networks (NNs) using a novel variant of the “Additively Preconditioned Trust-region Strategy” (APTS). The proposed method is based on a parallelizable additive domain decomposition approach applied to the neural network’s parameters. Built upon the TR framework, the APTS method ensures global convergence towards a minimizer. Moreover, it eliminates the need for computationally expensive hyper-parameter tuning, as the TR algorithm automatically determines the step size in each iteration. We demonstrate the capabilities, strengths, and limitations of the proposed APTS training method by performing a series of numerical experiments. The presented numerical study includes a comparison with widely used training methods such as SGD, Adam, LBFGS, and the standard TR method.
Published in , 1900
Enhancing training of physics-informed neural networks using domain-decomposition based preconditioning strategies.
We propose to enhance the training of physics-informed neural networks (PINNs). To this aim, we introduce nonlinear additive and multiplicative preconditioning strategies for the widely used L-BFGS optimizer. The nonlinear preconditioners are constructed by utilizing the Schwarz domain-decomposition framework, where the parameters of the network are decomposed in a layer-wise manner. Through a series of numerical experiments, we demonstrate that both, additive and multiplicative, preconditioners significantly improve the convergence of the standard L- BFGS optimizer, while providing more accurate solutions of underlying partial differential equations. Moreover, the additive preconditioner is inherently parallel, thus giving rise to a novel approach to model parallelism.
Published in , 1900
DeepONet Based Preconditioning Strategies for Solving Parametric Linear Systems of Equations.
We introduce a new class of hybrid preconditioners for solving parametric linear systems of equations. The proposed preconditioners are constructed by hybridizing the deep operator network, namely DeepONet, with standard iterative methods. Exploiting the spectral bias, DeepONet-based components are harnessed to address low-frequency error components, while conventional iterative methods are employed to mitigate high-frequency error components. Our preconditioning framework comprises two distinct hybridization approaches: direct preconditioning (DP) and trunk basis (TB) approaches. In the DP approach, DeepONet is used to approximate an action of an inverse operator to a vector during each preconditioning step. In contrast, the TB approach extracts basis functions from the trained DeepONet to construct a map to a smaller subspace, in which the low-frequency component of the error can be effectively eliminated. Our numerical results demonstrate that utilizing the TB approach enhances the convergence of Krylov methods by a large margin compared to standard non-hybrid preconditioning strategies. Moreover, the proposed hybrid preconditioners exhibit robustness across a wide range of model parameters and problem resolutions.
Published in , 1900
Multilevel Objective-Function-Free Optimization with an Application to Neural Networks Training.
A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multi-level feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behaviour in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.
Published in , 1900
A phase-field approach to conchoidal fracture.
Crack propagation involves the creation of new internal surfaces of a priori unknown paths. A first challenge for modeling and simulation of crack propagation is to identify the location of the crack initiation accurately, a second challenge is to follow the crack paths accurately. Phase-field models address both challenges in an elegant way, as they are able to represent arbitrary crack paths by means of a damage parameter. Moreover, they allow for the representation of complex crack patterns without changing the computational mesh via the damage parameter, which however comes at the cost of larger spatial systems to be solved. Phase-field methods have already been proven to predict complex fracture patterns in two and three dimensional numerical simulations for brittle fracture. In this paper, we consider phase-field models and their numerical simulation for conchoidal fracture. The main characteristic of conchoidal fracture is that the point of crack initiation is typically located inside of the body. We present phase-field approaches for conchoidal fracture for both, the linear-elastic case as well as the case of finite deformations. We moreover present and discuss efficient methods for the numerical simulation of the arising large scale non-linear systems. Here, we propose to use multigrid methods as solution technique, which leads to a solution method of optimal complexity. We demonstrate the accuracy and the robustness of our approach for two and three dimensional examples related to mussel shell like shape and faceted surfaces of fracture and show that our approach can accurately capture the specific details of cracked surfaces, such as the rippled breakages of conchoidal fracture. Moreover, we show that using our approach the arising systems can also be solved efficiently in parallel with excellent scaling behavior.
Published in , 1900
Nonlinear Schwarz preconditioning for nonlinear optimization problems with bound constraints.
We propose a nonlinear additive Schwarz method for solving nonlinear optimization problems with bound constraints. Our method is used as a “right preconditioner" for solving the first-order optimality system arising within the sequential quadratic programming (SQP) framework using Newton’s method. The algorithmic scalability of this preconditioner is enhanced by incorporating a solutiondependent coarse space, which takes into account the restricted constraints from the fine level. By means of numerical examples, we demonstrate that the proposed preconditioned Newton methods outperform standard active-set methods considered in the literature.
Published in , 1900
On the use of hybrid coarse-level models in multilevel minimization methods.
Solving large-scale nonlinear minimization problems is computationally demanding. Nonlinear multilevel minimization (NMM) methods explore the structure of the underlying minimization problem to solve such problems in a computationally efficient and scalable manner. The efficiency of the NMM methods relies on the quality of the coarse-level models. Traditionally, coarse-level models are constructed using the additive approach, where the so-called tau-correction enforces a local coherence between the fine-level and coarse-level objective functions. In this work, we extend this methodology and discuss how to enforce local coherence between the objective functions using a multiplicative approach. Moreover, we also present a hybrid approach, which takes advantage of both, additive and multiplicative, approaches. Using numerical experiments from the field of deep learning, we show that employing a hybrid approach can greatly improve the convergence speed of NMM methods and therefore it provides an attractive alternative to the almost universally used additive approach.
Published in , 1900
A Matrix-free Multigrid Preconditioner for Jacobian-free Newton-Krylov Methods.
In this work, we propose a multigrid preconditioner for Jacobian-free Newton-Krylov (JFNK) methods. Our multigrid method does not require knowledge of the Jacobian at any level of the multigrid hierarchy. As it is common in standard multigrid methods, the proposed method also relies on three building blocks: transfer operators, smoothers, and a coarse level solver. In addition to the restriction and prolongation operator, we also use a projection operator to transfer the current Newton iterate to a coarser level. The three-level Chebyshev semi-iterative method is employed as a smoother, as it has good smoothing properties and does not require the representation of the Jacobian matrix. We replace the direct solver on the coarsest-level with a matrix-free Krylov subspace method, thus giving rise to a truly Jacobian-free multigrid preconditioner. We will discuss all building blocks of our multigrid preconditioner in detail and demonstrate the robustness and the efficiency of the proposed method using several numerical examples.
Published in , 1900
<!– ## DeepONet Based Preconditioning Strategies for Solving Parametric Linear Systems of Equations.
A class of multi-level algorithms for unconstrained nonlinear optimization is presented which does not require the evaluation of the objective function. The class contains the momentum-less AdaGrad method as a particular (single-level) instance. The choice of avoiding the evaluation of the objective function is intended to make the algorithms of the class less sensitive to noise, while the multi-level feature aims at reducing their computational cost. The evaluation complexity of these algorithms is analyzed and their behaviour in the presence of noise is then illustrated in the context of training deep neural networks for supervised learning applications.
Published in , 1900
Multilevel Active-Set Trust-Region (MASTR) Method for Bound Constrained Minimization.
We introduce a novel variant of the recursive multilevel trust-region (RMTR) method, called MASTR. The method is designed for solving non-convex bound-constrained minimization problems, which arise from the finite element discretization of partial differential equations. MASTR utilizes an active-set strategy based on the truncated basis approach in order to preserve the variable bounds defined on the finest level by the coarser levels. This approach allows for fast convergence of the MASTR method, especially once the exact active-set is detected. The efficiency of the method is demonstrated by means of several numerical examples.
Published in , 1900
A detailed investigation of the model influencing parameters of the phase-field fracture approach.
Phase-field approaches to fracture are gaining popularity to compute a priori unknown crack paths. In this work the sensitivity of such phase-field approaches with respect to its model specific parameters, that is, the critical length of regular- ization, the degradation function and the mobility, is investigated. The susceptibility of the computed cracks to the setting of these parameters is studied for problems of linear and finite elasticity. Furthermore, the convergence properties of different solution strategies are analyzed. Monolithic and staggered solution schemes for the solution of the arising nonlinear discrete systems are studied in detail. To conclude, we demonstrate the versatility of the phase-field fracture approach in a real-world problem by comparing different simulations of conchoidal fracture using structured and unstructured meshes.
Published in , 1900
Nonlinear Field-split Preconditioners for Solving Monolithic Phase-field Models of Brittle Fracture.
One of the state-of-the-art strategies for predicting crack propagation, nucleation, and interaction is the phase-field approach. Despite its reliability and robustness, the phase-field approach suffers from burdensome computational cost, caused by the non-convexity of the underlying energy functional and a large number of unknowns required to resolve the damage gradients. In this work, we propose to solve such nonlinear systems in a monolithic manner using the Schwarz preconditioned inexact Newton's (SPIN) method. The proposed SPIN method leverages the field split approach and minimizes the energy functional separately with respect to displacement and the phase-field, in an additive and multiplicative manner. In contrast to the standard alternate minimization, the result of this decoupled minimization process is used to construct a preconditioner for a coupled linear system, arising at each Newton's iteration. The overall performance and the convergence properties of the proposed additive and multiplicative SPIN methods are investigated by means of several numerical examples. Comparison with widely-used alternate minimization is also performed and we show a reduction in the execution time up to a factor of $50$. Moreover, we also demonstrate that this reduction grows even further with increasing problem size and larger loading increments.
Published in , 1900
Large scale simulation of pressure induced phase-field fracture propagation using Utopia.
Non-linear phase field models are increasingly used for the simulation of fracture propagation problems. The numerical simulation of fracture networks of realistic size requires the efficient parallel solution of large coupled non-linear systems. Although in principle efficient iterative multi-level methods for these types of problems are available, they are not widely used in practice due to the complexity of their parallel implementation. Here, we present Utopia, which is an open-source C++ library for parallel non-linear multilevel solution strategies. Utopia provides the advantages of high-level programming interfaces while at the same time a framework to access low-level data-structures without breaking code encapsulation. Complex numerical procedures can be expressed with few lines of code, and evaluated by different implementations, libraries, or computing hardware. In this paper, we investigate the parallel performance of our implementation of the recursive multilevel trust-region (RMTR) method based on the Utopia library. RMTR is a globally convergent multilevel solution strategy designed to solve non-convex constrained minimization problems. In particular, we solve pressure-induced phase-field fracture propagation in large and complex fracture networks. Solving such problems is deemed challenging even for a few fractures, however, here we are considering networks of realistic size with up to 1000 fractures.
Published in , 1900
A phase-field approach to pneumatic fracture.
Phase-field models have been proven to be reliable methods for the sim- ulation of complex crack patterns and crack propagation. In this contribution we investigate the phase-field model in linear and finite elasticity and summarize the influences of model specific parameters. Furthermore, externally driven fracture pro- cesses, in particular in the context of pneumatic fracture, are examined in detail. The focus is on fracture induced by pressure and anisotropic crack growth. Besides the modeling, the solution process is analyzed by applying multilevel methods. Within a series of parametric studies and numerical examples the versatility of phase-field models is demonstrated.
Published in , 1900
Subdivision-based nonlinear multiscale cloth simulation.
Cloth simulation is an important topic for many applications in computer graphics, animation, and augmented virtual reality. The mechanical behavior of cloth objects can be mod- eled by the Kirchhoff–Love thin shell equations, which lead to large-scale, nonlinear, ill-conditioned algebraic equations. We propose to solve these nonlinear problems efficiently using the recursive multilevel trust region (RMTR) method. Our multilevel framework for cloth simulations is based on Catmull–Clark subdivision surfaces, which facilitates generation of the mesh hierarchy and also provides the basis for the finite element discretization. The prolongation and restriction operators are similarly constructed based on the subdivision rules. Finally, we leverage a reverse subdivision operator to transfer iterates from fine levels to coarser levels. The novel use of this fine-to-coarse operator provides a computationally efficient alternative to the least-square approach used elsewhere. Using the resulting RMTR variant, we present numerical examples showing a reduction in the number of iterations by several orders of magnitude when compared to a single-level trust region method.
Published in , 1900
Efficient identification of scars using heterogeneous model hierarchies.
Detection and quantification of myocardial scars are helpful for diagnosis of heart diseases and for personalized simulation models. Scar tissue is generally characterized by a different conduction of excitation. We aim at estimating conductivity-related parameters from endocardial mapping data. Solving this inverse problem requires computationally expensive monodomain simulations on fine discretizations. We aim at accelerating the estimation by combining electrophysiology models of different complexity.
Published in , 1900
Quantitative analysis of nonlinear multifidelity optimization for inverse electrophysiology.
The conductivity of cardiac tissue determines excitation propagation and is important for quantifying ischemia and scar tissue and for building personalized models. Estimating conductivity distributions from endocardial mapping data is computationally challenging due to the computational complexity of the monodomain equations describing the cardiac excitation. For computing a maximum posterior estimate, we investigate different algorithmic optimization approaches based on adjoint gradient computation: steepest descent, limited memory BFGS, and recursive multilevel trust region methods using mesh hierarchies or heterogeneous model hierarchies. We compare overall performance, asymptotic convergence rate, and pre-asymptotic progress on some examples in order to assess the benefit of multifidelity acceleration.
Published in , 1900
Recursive multilevel trust region method with application to fully monolithic phase-field models of brittle fracture.
The simulation of crack initiation and propagation in an elastic material is difficult, as crack paths with complex topologies have to be resolved. Phase-field approaches allow to simulate crack behavior without the need to explicitly model crack paths. However, the underlying mathematical model gives rise to a non-convex constrained minimization problem. In this work, we propose a recursive multilevel trust region (RMTR) method to efficiently solve such a minimization problem. The RMTR method combines global convergence properties of trust region methods with the optimality of multilevel methods. The solution process is accelerated by employing level dependent objective functions, minimization of which provides correction to the original/fine-level problem. In the context of the phase-field fracture approach, it is challenging to design efficient level dependent objective functions as a certain mesh resolution is required to resolve fracture bands. We introduce level dependent objective functions that combine fine level description of the crack path with the coarse level discretization. The overall performance and the convergence properties of the proposed RMTR method are investigated by means of several numerical examples in three dimensions.
Published in , 1900
Globally Convergent Multilevel Training of Deep Residual Networks.
We propose a globally convergent multilevel training method for deep residual networks (ResNets). The devised method can be seen as a novel variant of the recursive multilevel trust-region (RMTR) method, which operates in hybrid (stochastic-deterministic) settings by adaptively adjusting mini-batch sizes during the training. The multilevel hierarchy and the transfer operators are constructed by exploiting a dynamical system's viewpoint, which interprets forward propagation through the ResNet as a forward Euler discretization of an initial value problem. In contrast to traditional training approaches, our novel RMTR method also incorporates curvature information on all levels of the multilevel hierarchy by means of the limited-memory SR1 method. The overall performance and the convergence properties of our multilevel training method are numerically investigated using examples from the field of classification and regression.
talks
teaching
Teaching experience 1
Undergraduate course, University 1, Department, 2014
This is a description of a teaching experience. You can use markdown like any other post.
Teaching experience 2
Workshop, University 1, Department, 2015
This is a description of a teaching experience. You can use markdown like any other post.