The Learning Problem: Models, Learning, and Optimization

The learning problem refers to training a physical or machine learning model to replicate features of a dataset. The model can then make predictions or decisions about new, unseen data. The learning problem can also refer to using a model combined with optimization approaches to reason about the world around us or solve specific problems of interest.

Thus, we build upon our previous content in this unit and focus on models and optimization approaches to represent and solve engineering problems.

• Ordinary least squares (OLS) is a method for estimating the parameters of a linear regression model. It finds the values of the parameters that minimize the sum of the squares of the differences between the observed responses in the data and the responses predicted by the model. OLS is widely used in statistical analysis and has several desirable properties, such as being unbiased and having a minimum variance among all unbiased estimators.

• Linear programming is a mathematical technique for optimizing an objective function subject to constraints. It is used to find a linear objective function’s maximum or minimum value subject to linear equality or inequality constraints. Linear programming can model and solve many optimization problems in finance, manufacturing, engineering, and transportation.

• Dynamic progamming and heuristic optimization is a branch of computer science and mathematics that optimizes combinatorial objects, such as graphs, functions, and strings. It involves finding the best solution to a problem among a finite or countably infinite set of possible solutions. Combinatorial optimization has many applications in computer science, engineering, and other fields.