# Mathematical Methods I

## Lecture notes and videos

1. Vector and Matrices
• Linear combination of vectors, inner product, length, angle between vectors; Different views of Ax=b.
[Video-Lecture] [Lecture Notes]

2. Solving Ax=b, Linear Independence, Rank, Column, Row and Null Spaces.
• When can I solve Ax=b? Basis and Linear Independence; Matrix operations, symmetric, skew, triangular and diagonal matrices; Rank of a matrix, rank and linear independence; Vector spaces and subspaces, basis, span; A first look at the column space, nullspace and row space; Inner product spaces, Cauchy-Scwartz inequality.
[Video-Lecture] [Lecture Notes]

3. Rank, Column, Row and Null Spaces
• Rank of a matrix, rank and linear independence; Vector spaces and subspaces, basis, span; A first look at the column space, nullspace and row space; Inner product spaces, Cauchy-Scwartz inequality.
[Video-Lecture] [Lecture Notes]

4. Matrix Operations and Elementary Matrices
• Review of the column and row views of Ax=b; Matrix Multiplication, Elementary Matrices, Permutation Matrices, Exchanging Columns, Multiplication as a Sum of Rank 1 Matrices, Matrix Block Multiplication; Matrix Inversion, Cancellation Laws of Matrix Multiplication, Inverse and Solution of Ax=b; Summary of Matrix Operations
[Video-Lecture] [Lecture Notes]

5. Matrix Elimination
• Matrix Elimination, the augmented matrix, elimination example, elimination matrices and their inverses, elimination with row exchange; A=LU factorization.
[Video-Lecture] [Lecture Notes]

6. LU Factorization, Matrix Inversion, Gauss-Jordan Form
• A=LU factorization, A=LDU factorization, forward elimination and back substitution, Cost of factorization, Permutation matrices and row exchanges, row equivalent systems, Schur Complement; Row Echelon form and when does elimination fails? Inverse of a matrix, Gauss – Jordan form.
[Video-Lecture] [Lecture Notes]

7. Vector Spaces and Subspaces, Basis and Dimension
• Spaces of vectors, the space ℝn, vector space, examples of vector spaces, the zero vector; Subspace, union of subspaces, subspaces of ℝ3, examples; The column space of A, Subspace spanned by a set of vectors, examples; The null space of A, solving Ax=0 and Rx=0, examples, pivot columns and free columns, the reduced row echelon form, solving Ax=0 and special solutions, computing the nullspace; The rank of a matrix, rank 1 matrices, rank as the dimension of C(A); Matrix elimination summary.
[Video-Lecture] [Lecture Notes]

8. Lecture 8;
• The column space of A; The null space of A; The row space of A; The nullspace of AT; Solving Ax=0; Rank of a matrix; The complete solution to Ax=b; Independence, basis and dimension; Dimensions of the four fundamental spaces
[Video-Lecture] [Lecture Notes]

9. Independence, Basis, and Dimension
• Independence, basis and dimension, vectors that span a subspace, row and column spaces, bases for the column and row spaces, dimension of a vector space, dimensions of C(A) and N(A), bases for matrix and function spaces; Summary – independence, basis and dimension.
[Video-Lecture] [Lecture Notes]

10. Dimensions of the Four Fundamental Subspaces
• The four subspaces of A, C(A), N(A), C(AT), N(AT); The dimensions of the column, row, nullspace and left null spaces; Applications.
[Video-Lecture] [Lecture Notes]

11. The Four Fundamental Subspaces: Applications to Graphs and Orthogonality
• An example of the fundamental subspaces from graphs, networks and electrical circuits; Example in rank 1 matrices and in rank 2 matrices; Summary of dimensions and bases of the four fundamental subspaces; Inner Product Spaces, Orthogonality, Orthogonality of the four fundamental subspaces, Orthogonal complements, Combining spaces.
[Video-Lecture] [Lecture Notes]
12. Projections and Least Squares
• Projections matrices, Projection onto a line, Projection onto a subspace, Normal equations of least squares.
[Video-Lecture] [Lecture Notes]

13. Projections and Least Squares (Continued)
• Projections matrices, Projection onto a line, Projection onto a subspace, Summary; Least Squares approximation, Minimizing the error, Geometric presentation of least squares, Regression – Fitting a straight line, Fitting a parabola, Summary; Orthogonal basis and Gram-Schmidt orthogonalization, Projections using an orthonormal basis, The Gram-Schmidt Decomposition, The 𝑨=𝑸𝑹 factorization, Summary.
[Video-Lecture] [Lecture Notes]

14. Least Squares, Orthogonal Matrices, Gram-Schmidt and A=QR Factorization (Continued)
• Reviewing projection onto a subspace, Least Squares approximation, Minimizing the error, Linear regression; Orthogonal basis and Gram-Schmidt orthogonalization, Projections using an orthonormal basis, The A=QR factorization, Summary.
[Video-Lecture] [Lecture Notes]

15. The Properties of Determinants

• detI=1, sign reversal and linearity in each row; Elimination and determinant; Invertable matrices and determinant; The determinant of AB and of AT; Permutations and cofactors; The Pivot Formula.
[Video-Lecture] [Lecture Notes]

16. Determinant Formulas, Cofactors, Cramer's Rule, Inverse Matrix, Area and Volume
• Formulas for evaluating the determinant; Cofactor formula; Evaluating A-1; Cramer's rule for x=A-1b; Area and Volume as Determinants.
[Video-Lecture] [Lecture Notes]

17. Introduction to Eigenvalues and Eigenvectors
• Introduction to Eigenvalues and Eigenvectors; Determining eigenvalues and eigenvectors; Applications to Symmetric and Orthogonal Matrices; The Eigenvalue Problem for Projection and Reflection Matrices; Introduction to Diagonalization of a Matrix.
[Video-Lecture] [Lecture Notes]
18. Diagonalization of a Matrix, Matrix Exponential
• Distinct Eigenvalues, Independent Eigenvectors and Diagonalization of a General Matrix; Examples of Diagonalizable and Non-Diagonalizable Matrices; Markov Matrices; Similar Matrices; Example of Fibonacci Numbers; Geometric and Algebraic Multiplicity.
[Video-Lecture] [Lecture Notes]

19. Differential Equations and the Eigenvalue Problem, Exponential of a Matrix, Diagonalization of Symmetric Matrices
• System of Differential Equations, Difference Equations, Stability; Exponential of a Matrix, Examples; Symmetric and Anti-Symmetric Matrices, Diagonalization and Spectral Theorem; Repeated eigenvalues and Diagonalization; Schnur Theorem.
[Video-Lecture] [Lecture Notes]

20. Positive Definite Matrices and the Jordan Form
• Positive Definite Matrices; Tests for positive definite matrices - eigenvalues, pivots, sub-determinants, energy functional, ATA form; Positive Semi-Definite Matrices; Test for a Minimum of Energy; Quadratic form, transformation to principal axes; Similar matrices and Jordan form; Similarity Transformation.
[Video-Lecture] [Lecture Notes]

21. Singular Value Decomposition
• Low rank image representation; Eigenvectors for the SVD; Singular Values; Proof of the SVD; Examples of the SVD; Accounting for the nullspaces; Singular Value Stability versus Eigenvalue Stability.
[Video-Lecture] [Lecture Notes]

22. Linear Transformations
• Linear Transformations, Matrix Representation; Basis; Examples of linear transformations; Range and Kernel; Product of transformations; Choosing the best bases; Jordan form; Bases for Function spaces.
[Video-Lecture] [Lecture Notes]

23. Introduction to 1st Order Ordinary Differential Equations
• Differential Equation Models; Ordinary and Partial Differential Equations; Classification of ODEs; Direction fields and Equilibrium solutions; Solutions of simple ODEs; Initial Value Problems; Method of Integrating factors; Integrating factors for general 1st Order ODEs.
[Video-Lecture] [Lecture Notes]

24. 1st Order Ordinary Differential Equations
• Direction fields and Equilibrium solutions; Solutions of ODEs; Separable ODEs, Method of Integrating Factors; Exact ODEs.
[Video-Lecture] [Lecture Notes]

25. Linear Vs. Nonlinear ODEs, Autonomous Eqs and Population Dynamics, Existence and Uniqueness of Solutions
• Difference between linear and nonlinear Equations; Autonomous Equations and Population Dynamics; Numerical Approximations, Euler Method; Existence and Uniqueness of Solutions for linear and nonlinear 1st Order ODEs.
[Video-Lecture] [Lecture Notes]

26. 2nd Order Differential Equations
• Second-Order Linear Homogeneous Equations with Constant Coefficients, Solutions of Linear Homogeneous Equations; The Wronskian; Complex Roots of the Characteristic Equation, Repeated Roots; Reduction of Order, Nonhomogeneous Equations; Method of Undetermined Coefficients, Variation of Parameters.
[Video-Lecture] [Lecture Notes]

27. Mechanical Vibrations
• Variation of Parameters; Undamped free vibrations; Damped free vibrations; Critical damping value; Forced vibrations with damping; Transient and Steady-State Solutions; Maximum amplitude and resonance; Undamped Equatiion and Beats.
[Video-Lecture] [Lecture Notes]

28. Power Series Methods
• Review of Power Series; Series Solutions near ordinary points; Euler equations and regular singular points.
[Video-Lecture] [Lecture Notes]

29. Series Solutions of ODEs, Bessel Functions

30. Laplace Transform and ODEs

31. Bessel's Equation
• Bessel's equation and solutions; Bessel Equation of Order Zero; Bessel Equation of Order One-Half; Bessel Equation of Order One.
[Video-Lecture] [Lecture Notes]

32. Introduction to the Laplace Transform
• Definition of the Laplace transform; Solution of Initial Value Problems; Examples of Elementary Laplace Transforms; Step Functions.
[Video-Lecture] [Lecture Notes]

33. Differential Equations and the Laplace Transform
• Differential equations with discontinuous forcing terms; Solution smoothness; Impulse Functions and impulse response; The convolution integral, transfer function.
[Video-Lecture] [Lecture Notes]

34. Two-Point Boundary Value Problems and Fourier Series
• Two-point boundary value problems; Fourier Series; The Fourier Convergence Theorem; Gibbs Phenomenon.
[Video-Lecture] [Lecture Notes]

35. The Fourier Convergence Theorem, Separation of Variables and the Heat Diffusion Equation
• The Fourier Convergence Theorem; Even and Odd Functions; Even and odd periodic extensions of functions; Separation of variables for the solution of hear conduction problems; Eigenvalues, eigenvectors and Fourier expansion; Solving boundary value problems with essential, natural and mixed boundary conditions.
[Video-Lecture] [Lecture Notes]

36. The Wave Equation and the Laplace Equation
• Vibrations of an elastic string with a non-zero initial displacement; Fundamental modes; Elastic String with a non-zero initial velocity; Laplace's equation, Dirichlet and Neumann problems; Dirichlet problem in a rectangular and a circle.
[Video-Lecture] [Lecture Notes]

37. Introduction to Sturm-Liouville Theory
• Generalization of two-point boundary value problems; Sturm-Liouville Boundary Value Problems; Lagrange's identity; Real eigenvalues and eigenfunctions, orthogonality, orthonornal eigenfunctions, examples; Eigenfunction expansions, convergence theorem; Sturm-Liouville problems and algebraic eigenvalue problems; Self-adjoint problems.
[Video-Lecture] [Lecture Notes]

38. Sturm-Liouville Theory
• Eigenfunction expansion, Convergence theorem, Examples; Sourm-Liouville problems, Algebraic eigenvalue problems and Linear Operator Theory; Self-adjoint problems; Non-homogeneous boundary value problems, Eigenfunction expansion, Formal solution to the non-homogeneous problem, Fredholm alternative, examples; Solving the non-homogeneous transient heat conduction problem, speed of convergence of the solution.
[Video-Lecture] [Lecture Notes]

39. Singular Sturm-Liouville Problems, Approximation Theory
• Singular Sturm-Liouville problems, Examples; Bessel series expansion, Vibrations of a circular elastic membrane; Pointwise and mean convergence, mean square error, approximation theory, square integrable functions.
[Video-Lecture] [Lecture Notes]

40. System of ODEs
• Basic Theory of Systems of First-Order Linear Equations; Homogeneous Linear Systems with Constant Coefficients; Complex Eigenvalues.
[Video-Lecture] [Lecture Notes]

41. System of ODEs, Phase Plane and Qualitative Methods
• Linear System of ODEs; Critical Points for Different Cases; Real and Complex Eigenvalues; Repeated Eigenvalues; Solution of Homogeneous and Non-Homogeneous Systems of ODEs; Fundamental Matrices.
[Video-Lecture] [Lecture Notes]

42. Linear ODE Systems and Stability, Non-linear ODE Systems
• Criteria for critical points, Stability; Non-linear systems of ODEs, Linearization and Stability.
[Video-Lecture] [Lecture Notes]

43. Nonlinear Differential Equations and Stability
• The Phase Plane, Linear Systems; Autonomous Systems and Stability; Locally Linear Systems; Competing Species, Predator–Prey Equations.
[Video-Lecture] [Lecture Notes]

## Homework

• September 1, Homework 1
• Linear Independence, Elimination, LU Factorization, Gauss-Jordan and the Inverse of a Matrix, Nullspace and Column Space
[Homework] [Solution]

• September 16, Homework 2
• September 24, Homework 3
• Least Squares, Orthogonality, Gram-Schmidt and QR Factorization
[Homework] [Solution]

• October 2, Homework 4
• Properties of Determinants, Orthogonality, Eigenvalues and Eigenvectors, Diagonalization
[Homework] [Solution]

• October 8, Homework 5
• Eigenvalues and Eigenvectors, Positive Definite Matrices, Quadratic Forms, SVD
[Homework] [Solution]

• October 15, Homework 6
• Jordan Matrices, Similar Matrices, Linear Transformations, 1st Order ODEs
[Homework] [Solution]

• October 29, Homework 7

• November 8, Homework 8

• October 15, Homework 6
• Jordan Matrices, Similar Matrices, Linear Transformations, 1st Order ODEs
[Homework] [Solution]

• October 29, Homework 7

• November 8, Homework 8

• November 18, Homework 9

• November 26, Homework 10

• ## Exams

• September 27, Exam 1

• November 1, Exam 2
• Linear Algebra and Differential Equations
[Exam] [Solution]

• December 6, Exam 3
• Linear Algebra and Differential Equations
[Exam] [Solution]

## Course info and references

Credit: 3 Units

Lectures: MWF, 10:30 -- 11:20 am, DeBartolo Hall 215.

Professor: Nicholas Zabaras, 311 I Cushing Hall, nzabaras@gmail.com

Teaching Assistants: Zhuogang Pengzpeng5@nd.eduSamaresh Midyasmidya@nd.edu

Office hours: Teaching Assistants, Mondays (and if there is demand Wednesdays) 5:15 -- 6:15 p.m. (DeBartolo 203); N. Zabaras, Tuesdays 5:00-6:00 pm (Cushing 311 I).

Course description: This course provides a rigorous review of linear algebra including vector spaces and subspaces, eigenvalues and eigenvectors, orthogonality, singular value decomposition, linear transformations; Applications of linear algebra to networks, structures, estimation, Lagrange multipliers and optimization will be discussed. Also covered are: differential equations of equilibrium; Laplace's equation and potential flow; boundary-value problems and Sturm-Liuville Theory; system of linear ODEs, non-linear differential equations and stability; Fourier series; discrete Fourier transform; convolution; and applications.

Goals for the Course: The purpose of the course is to provide you useful insights and understanding of mathematical methods that will be relevant to your research work. While the complexity of calculations will be kept to a minimum, we will put emphasis on integrating concepts from linear algebra to understand the nature of the solutions of systems of linear and non-Linear differential equations. Our goal is to identify the underlying patterns in many applications and allow clear understanding of the nature of the solutions of equations and development of efficient methods to compute them.

Intended audience: Graduate Students in Engineering and the Sciences. Qualified undergraduates can also be enrolled.

References of General Interest: The course lecture slides will become available on the course web site. Please note that the lecture slides are to be read on the internet - and not to be printed. For in depth study, readings from the recommended textbooks will be highlighted. While there is no required text for this course, most of the lectures will follow closely the books by G. Strang and W.E. Boyce and R.C. DiPrima. The books by E. Kreysig and M. Greenberg provide comprehensive material for all lectures (and beyond) and can be useful for subsequent engineering Mathematics courses.

Recommended Textbooks:

Homework: Assigned every week. We anticipate approximately ten homework sets. While ideally all homeworks should be submitted typed to allow easy readability, as a minimum you will need to scan your work and submit it electronically. Your homework solutions should be accompanied by any computer programs (e.g. MatLab scripts, data files, Readme files, etc.) and mailed by midnight of the due date tothis Email address. All attachments should arrive on an appropriately named zipped directory (e.g. HW1_Submission_YourName.rar). We will be asking for volunteers weekly to prepare Latex solutions to each homework set. We much appreciate your help.

Exams: There will be three midterm take-home exams (no final). The designated days for the exam are as follows: September 27th, November 2nd and December 6th. The exams are two hrs long but you will be provided freedom within 24 hrs of selecting your start. Managing of downloading the exams and their solution within 2 hrs will be provided through Sakai. Note that we reserve the right to change the format of the exams if it becomes clear that we cannot guarantee the university's honor code.

Grading: Homework 10% and each exam 30%.

Prerequisites: Elementaty Linear Algebra, Differential Calculus, Previous exposure to differential equations. Some programming background in MatLab (or if you desire in Mathematica, Python, etc.) is required for some of the homework. Also it will be helpful to acquire Latex skills in assisting with the homework solution preparation.

Honor Code: Students are expected to understand and abide by the principles and procedures set forth in the University of Notre Dame Academic Code of Honor and uphold the pledge that "As a member of the Notre Dame community, I acknowledge that it is my responsibility to learn and abide by principles of intellectual honesty and academic integrity, and therefore I will not participate in or tolerate academic dishonesty". Students may collaboratively discuss course assignments, but are expected to write and complete their own assignments independently. Downloading solutions from the web or other sources is not allowed. Finally, there should be absolutely no interactions between students regarding the takehome exams.

## Syllabus

1. Introduction to Vectors and Matrices
• Lengths and Dot Products
• Linear Combinations of Vectors
2. Linear Equations
• Elimination using Matrices
• Inverse Matrices, Transposes and Permutations
• A=LU Factorization
3. Vector Spaces and Subspaces
• Spaces of Vectors
• The nullspace of A
• Independence, Basis, Dimension
• The Four Fundamental Subspaces
• Orthogonality
• Projections and Least Squares
• Orthonormal Bases and Gram-Schmidt
4. Eigenvalues and Eigenvectors
• Properties of Determinants, Cramer's Rule and Inverses
• Eigenvalue Problem
• Diagonalizing a Matrix
• Symmetric and Positive Definite Matrices
• Applications to Systems of ODEs
5. Singular Value Decomposition
• Fundamentals of SVD
• Principal Component Analysis
• Geometry of SVD
6. Linear Transformations
• Matrix Representation
• Optimal Basis Representation
7. Applications of Linear Algebra
• Hermitian and Unitary Matrices, Fast Fourier Transform
• Linear Algebra for Functions
• Graphs and Networks
• Markov Matrices
• Linear Programming
8. First-Order Differential Equations
• Integrating Factors
• Numerical Approximations
• Existence and Uniqueness of Solution
• First Order Difference Equations
9. Second-Order Linear Equations
• Solutions of Linear Homogeneous Equations
• Reduction of Order
• Method of Undetermined Coefficient
• Variation of Parameters
• Forced Vibrations
10. Series Solutions of 2nd Order Linear Equations
• Power Series Fundamentals
• Series Solutions Near an Ordinary Point
• Euler Equations
• Series Solutions Near a Regular Singular Point, Bessel's Equation
11. The Laplace Transform
• Solution of Initial Value Problems
• Differential Equations, Forcing Functions, Impulse Functions
• Convolution
12. Systems of First Order Linear Equations
• Homogeneous Linear Systems with Constant Coefficients
• Nonhomogeneous Linear Systems
13. Nonlinear Differential Equations and Stability
• The Phase Plane, Linear Systems and Stability
• Locally Linear Systems
• Predator-Prey Equations
• Liapunov's Second Method
• Limit Cycles
• Chaos and Strange Attractors, The Lorenz Equations
14. Introduction to Partial Differential Equations
• Fourier Series and The Fourier Convergence Theorem
• Separation of Variables; Heat Conduction in a Rod
• The Wave Equation: Vibrations of an Elastic String
• Laplace’s Equation
15. Boundary Value Problems and Sturm-Liuville Theory
• Sturm–Liouville Boundary Value Problems
• Nonhomogeneous Boundary Value Problems
• Singular Sturm–Liouville Problems
• Bessel Series Expansion
• Series of Orthogonal Functions: Mean Convergence