Modern data often consists of feature vectors with a large number of features. High-dimensional geometry and Linear Algebra (Singular Value Decomposition) are two of the crucial areas which form the mathematical foundations of Data Science. This mini-course covers these areas, providing intuition and rigorous proofs. Connections between Geometry and Probability will be brought out. Text Book: Foundations of Data Science.