Title: | Constrained Matrix Factorization and Tensor Decomposition for Machine Learning |
Speaker: | Dr. Junjun Pan, Department of Mathematics, Hong Kong Baptist University, Hong Kong |
Time/Place: | 11:00 - 12:00 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Matrix factorization and tensor decomposition are important techniques in data analysis and have been widely used in machine learning for tasks such as dimensionality reduction, feature extraction, clustering, etc. They allow us to extract meaningful information from large and complex datasets. In this presentation, I will discuss our recent works on constrained matrix factorization and tensor decomposition, built based on the properties of tasks and data types. The first part of the presentation will focus on separable matrix factorization used in feature selection. Inspired by the interpretation of the separability in text data, and based on the standard separable nonnegative matrix factorization (NMF), we propose two more adaptable models to determine the underlying features. One is called generalized separable NMF and the other one is Co-separable NMF. We will present the key ideas and algorithms behind each model and their applications in text and facial datasets. The second part of the presentation will focus on constrained tensor Tucker decomposition. In the clustering task for high-dimensional data, to improve the interpretability and sparsity of the model, we propose a model called orthogonal nonnegative Tucker decomposition, which is based on the standard Tucker decomposition by introducing orthogonality and nonnegativity constraints on the factor matrices. The motivation and algorithms behind the model will be demonstrated. |
Title: | PhD Oral Defense: Change Point Detection and Order Determination for Complex Data |
Speaker: | Mr ZHAO Wenbiao, Department of Mathematics, Hong Kong Baptist University, Hong Kong |
Time/Place: | 16:30 - 18:30 FSC1217, or Zoom (Meeting ID: 944 5990 3530 Password: 620255) |
Abstract: | The identification of change points is a central problem in statistical inference with critical implications for numerous applications. Similarly, the issue of dimension selection is a critical problem in statistical and machine learning research, with far reaching implications for various fields. This thesis presents an innovative approach for detecting change points in high-dimensional data and symmetric positive definite matrices, as well as a new method for dimension selection of tensor decomposition and envelope models. These approaches are carefully constructed with emphasis on their performance and their asymptotic properties, which are presented along with the corresponding estimation consistency. To evaluate their effectiveness, we conduct various numerical simulations and applications. In this presentation, I would first introduce the detection of multiple change points in high-dimensional data while avoiding the computational intensity that comes with exhaustive search algorithms and the challenge of controlling false positives that arises with hypothesis testing-based methods. Our technique utilizes a signal statistic, which is based on a sequence of signal screening-based local U-statistics and can handle sparse or dense data structures, with exponential dimensionality relative to the sample size. We establish the estimation consistency of the approach, even when the number of change points diverges at a certain rate. Additionally, we show that our method can effectively identify the change point locations due to its visualization nature, and we illustrate its performance through numerical studies and a real data example. Secondly, I would introduce a novel approach to identify multiple change points in random symmetric positive definite matrices, which are elements of a Riemannian manifold. We achieve this through MOSUM-based signal statistics, utilizing a matrix-log mean model with heterogeneous noise to model the data and capture the underlying manifold structure. The proposed approach is theoretically shown to consistently recover all change points, and simulation studies further demonstrate its effectiveness in detecting change patterns in functional connectivity. Lastly, I would propose a general method for dimension selection in various tensor decomposition and envelope regression models, which require identifying a low-rank tensor structure. Consistently selecting the structural dimensions or tensor ranks is an important issue in both theory and practice. Our approach addresses this challenge and can be applied to a wide range of models. We demonstrate its effectiveness through numerical simulations. Overall, this presentation introduces innovative approaches to critical problems in statistical inference and machine learning, along with their asymptotic properties and estimation consistency. The proposed methods show promising results in various numerical simulations and real data applications, highlighting their effectiveness in practice. |
Title: | The Upper-crossing/solution (US) Algorithm For Root-finding With Strongly Stable Convergence |
Speaker: | Professor Guoliang Tian, Department of Statistics and Data Science, Southern University of Science and Technology, China |
Time/Place: | 15:30 - 16:30 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | In this paper, we propose a new and broadly applicable root-finding method, called as the upper-crossing/solution (US) algorithm, which belongs to the category of non-bracketing (or open domain) methods. The US algorithm is a general principle for iteratively seeking the unique root θ^* of a non-linear equation g(θ) = 0 and its each iteration consists of two steps: an upper-crossing step (U-step) and a solution step (S-step), where the U-step finds an upper-crossing function or a U-function U(θ|θ^((t))) [whose form depends on θ^((t)) being the t-th iteration of θ^*] based on a new notion of so-called changing direction inequality, and the S-step solves the simple U-equation U(θ│θ^((t) ) )=0 to obtain its explicit solution θ^((t+1)). The US algorithm holds two major advantages: (i) It strongly stably converges to the root θ^*; and (ii) it does not depend on any initial values, in contrast to Newton's method. The key step for applying the US algorithm is to construct one simple U-function U(θ|θ^((t))) such that an explicit solution to the U-equation U(θ│θ^((t) ) )=0 is available. Based on the first-, second- and third-derivative of g(θ), three methods are given for constructing such U-functions. We show various applications of the US algorithm in calculating quantile in continuous distributions, calculating exact p-values for skew null distributions, and finding maximum likelihood estimates of parameters in a class of continuous/discrete distributions. The analysis of the convergence rate of the US algorithm and some numerical experiments are also provided. Especially, because of the property of strongly stable convergence, the US algorithm could be one of the powerful tools for solving an equation with multiple roots. |
Title: | Online Nonparametric Estimation for Streaming Data |
Speaker: | Professor Fang Yao, School of Mathematical Sciences, Peking University, China |
Time/Place: | 16:30 - 17:30 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Online learning and modeling have attracted considerable interest due to increasingly available data in streaming manner. Nonparametric models, although flexible, have seen limited use in online settings due to their data-driven nature and high computational demands. We introduce an innovative online method for dynamically updating local polynomial regression estimates. Our approach decomposes kernel-type estimates into two sufficient statistics and approximates future optimal bandwidths with a dynamic candidate sequence. This idea extends to general nonlinear optimization problems, where we propose an online smoothing backfitting algorithm for generalized additive models (GAM). We establish asymptotic normality and efficiency lower bounds for online estimation, shedding light on the trade-off between accuracy and computational cost driven by the bandwidth sequence length. For GAM, we also investigate statistical and algorithmic convergence and provide a framework for balancing estimation and computation performance. Our proposed online estimation is also applicable to complex structural data such as functional data. Simulations and real data examples are provided to support the usefulness of the proposed method. |
Title: | A Search in the Splitting of Exponential Operators and their Errors for Multiphysics and Multibody Dynamics Applications |
Speaker: | Prof. Qin Sheng, Department of Mathematics, Faculty of Science, Hong Kong Baptist University, Hong Kong |
Time/Place: | 11:00 - 12:00 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Exponential operator decompositions, associated with advanced numerical methods and adaptations, have been widely used in multiphysics and multibody applications. They have demonstrated powerful capabilities for solving major classes of mathematical modeling equations including nonlinear different differential equations in dynamical applications. Latest challenging research issues in the areas ranging from decompositions for higher accuracy and flexibility in parallel computation environments, splitting for non-linear and singular partial differential equations, deep neural network configurations overcoming the curse of dimensionality, non-linear stability and conservations of key physical components, iterative and adaptive strategies with specific applications in medical treatment plans, to quantum exponential decompositions in modern technology breakthroughs. It will provide a quick review of the pioneering work by Dr. M. Suzuki et al, and concentrate on error estimates of the splitting formulas in terms of computational applications in multiphysics and multibody dynamics. The talk will be suitable for all students who are interested in scientific computing. |
Title: | The Interplay Between Optimization On Matrix Manifolds And Matrix Eigenvalue Problems |
Speaker: | Prof. BAI Zhaojun, University of California |
Time/Place: | 11:00 - 12:30 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Optimization on matrix manifolds (OptMM) are ubiquitous in scientific computing, such as energy minimization in electronic structure calculations, finding projection matrices for dimensionality reduction in high dimensional data analysis, and robustification for the effects of data variability. In this talk, I will provide a perspective on intriguing interplay between OptMM and matrix eigenvalue problems (EVP). We will show how to exploit underlying properties of OptMM and EVP to reveal variational characterization of EVP, and design efficient numerical algorithms for solving large scale OptMM with EVP. The success of the perspective will be shown in solving challenging problems in real-life applications, such as low-rank approximation of tensor networks, robust common spatial pattern analysis in brain-computer interfaces. |
Title: | Understanding Hierarchical Representations in Deep Networks via Intermediate Features |
Speaker: | Dr. Peng Wang, Department of Electrical Engineering and Computer Science, University of Michigan |
Time/Place: | 11:00 - 12:00 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Over the past decade, deep learning has proven to be a highly effective method for learning meaningful features from raw data. This work attempts to unveil the mystery of hierarchical feature learning in deep networks by investigating the structures of intermediate features. Specifically, in the context of multi-class classification problems, we explore how deep linear networks transform input data into output by investigating the output (i.e., features) of each layer after training. Towards this goal, we first define metrics to measure within-class compression and between-class discrimination of intermediate features, respectively. Through an analysis of these two metrics, we show that the evolution of features follows a simple and quantitative pattern from shallow to deep layers: Each layer of linear networks progressively compresses within-class features at a geometric rate and discriminates between-class features at a linear rate with respect to layer index. To the best of our knowledge, this is the first quantitative characterization of feature evolution in hierarchical representations of deep networks. Moreover, our extensive experiments not only validate our theoretical results numerically but also reveal a similar pattern in deep nonlinear networks which aligns well with recent empirical studies. |
Title: | Dynamic Logistic State Space Prediction Model For Clinical Decision Making |
Speaker: | Dr. Jiakun Jiang, Beijing Normal University, China |
Time/Place: | 15:30 - 16:30 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Prediction modeling for clinical decision making is of great importance and needed to be updated frequently with the changes of patient population and clinical practice. Existing methods are either done in an ad hoc fashion, such as model recalibration or focus on studying the relationship between predictors and outcome and less so for the purpose of prediction. In this article, we pro- pose a dynamic logistic state space model to continuously update the parameters whenever new information becomes available. The proposed model allows for both time-varying and time-invariant coefficients. The varying coefficients are modeled using smoothing splines to account for their smooth trends over time. The smoothing parameters are objectively chosen by maximum likelihood. The model is updated using batch data accumulated at prespecified time intervals, which allows for better approximation of the underlying binomial density function. In the simulation, we show that the new model has significantly higher prediction accuracy compared to existing methods. We apply the method to predict 1 year survival after lung transplantation using the United Network for Organ Sharing data. |
Title: | Mathematics in Exploration Geophysics: Nonlinear dynamic models on vibrations in rotary drilling systems |
Speaker: | Dr. Joseph Ma, Halliburton, USA |
Time/Place: | 10:00 - 11:00 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Drilling serves as the first step of intrusive exploration for materials in deep Earth. Vibration control at the drill bit is an important part, both to ensure geomechanical stability and drill trajectory accuracy. Fast geometric computation is also required for automated control systems respond. In this seminar, we will explore the equation of motions involved in a rotary drilling system. Beginning with a two degrees-of-freedom self-excited vibration model, we will extend complexity to rock interaction model that involved bit characterization, cutter projection and bit-rock engagement correlation. Mathematical techniques on solving these nonlinear differential equations, and linear stability analysis of models will be presented. Finally, a verification on how such models can predict special oscillations in the coupled dynamics will be illustrated. |
Title: | Adaptive Functional Principal Components Analysis |
Speaker: | Mr. Sunny Wang, ENSAI, CREST, France |
Time/Place: | 16:00 - 17:00 FSC1217, Fong Shu Chuen Library, HSH Campus, Hong Kong Baptist University |
Abstract: | Functional data analysis (FDA) almost always involves smoothing discrete observations into curves, because they are never observed in continuous time and rarely without error. Although smoothing parameters affect the subsequent inference, data-driven methods for selecting these parameters are not well-developed, frustrated by the difficulty of using all the information shared by curves while being computationally efficient. On the one hand, smoothing individual curves in an isolated, albeit sophisticated way, ignores useful signals present in other curves. On the other hand, bandwidth selection by automatic procedures such as cross-validation after pooling all the curves together quickly become computationally unfeasible due to the large number of data points. In this paper we propose a new data-driven, adaptive kernel smoothing method, specifically tailored for functional principal components analysis (FPCA) through the derivation of sharp, explicit risk bounds for the eigen-elements. The minimization of these quadratic risk bounds provide refined, yet computationally efficient bandwidth rules for each eigen-element separately. Both common and independent design cases are allowed. Rates of convergence for the adaptive eigen-elements estimators are derived. An extensive simulation study, designed in a versatile manner to closely mimic characteristics of real data sets, support our methodological contribution, which is available for use in the R package FDAdapt. If time permits, natural extensions such as the adaptive estimation of functional scores will be discussed. |
We organize conferences and workshops every year. Hope we can see you in future.
Learn MoreProf. M. Cheng, Dr. Y. S. Hon, Dr. K. F. Lam, Prof. L. Ling, Dr. T. Tong and Prof. L. Zhu have been awarded research grants by Hong Kong Research Grant Council (RGC) — congratulations!
Learn MoreFollow HKBU Math