Documentation

You can enter your optimization problem into the code editor. If you press the 'Download Solver'-button, Python code is generated automatically that can solve this class of optimization problems. Some random sample data is generated as well, so you can run this code right away. The generated code provides functions that can compute the function values and derivatives of the objective function and the constraints. These functions are either fed into the SLSQP solver from the SciPy package or into the GENO solver. We suggest using the GENO solver as it is much faster and scales to problems involving thousands of variables and constraints. The GENO solver can be installed using the 'pip install genosolver' command.

The GENO Modeling Language

Consider for instance the following optimization problem:

\[ \begin{array}{rl} \displaystyle \min_{x} & c^\top \cdot x \\ \mbox{st} & A\cdot x \leq b \\ & x \geq 0, \end{array} \]

where \(A, b, c\) are given parameters and \(x\) is the optimization variable. This problem can be written in GENO as:

parameters
   matrix A
   vector b, c
variables
   vector x
min
   c'*x
st
   A*x <= b
   x >= 0

Details

A GENO specification has four blocks:

Declaration of the problem parameters that can be of type matrix, vector, or scalar,
declaration of the optimization variables that also can be of type matrix, vector, or scalar,
specification of the objective function in a MATLAB-like syntax, and finally
specification of the constraints, also in a MATLAB-like syntax that supports the following operators and functions: +, -, *, /, .*, ./, ^, .^, log, exp, sin, cos, tanh, abs, norm1, norm2, sum, tr, det, inv.

Input matrices can be specified as symmetric, psd (positive semidefinite), or sparse. The constraints section can also be omitted.

Examples

The following examples illustrate the GENO modeling language:

Least Squares Regression

parameters
   matrix A
   vector b
variables
   vector x
min norm2(A*x-b)^2

Non-negative Robust Regression

parameters
   matrix A
   vector b
variables
   vector x
min norm1(A*x-b)
st x >= 0

Symmtric, Non-negative Matrix Factorization

parameters
   Matrix X symmetric
variables
   Matrix U
min
   norm2(X - U*U')^2
st
   U >= 0

Quadratic Function over the Unit Simplex

parameters
   matrix Q psd
   vector c
variables
   vector x
min 0.5*x'*Q*x + c'*x
st sum(x) == 1
   x >= 0

The GENO Solver

The GENO solver combines an Augmented Lagrangian approach with a limited memory quasi-Newton method (L-BFGS-B) that can handle also bound constraints on the variables. Quasi-Newton methods are very efficient for problems involving thousands of optimization variables. The GENO solver is then instantiated by the automatically generated methods for computing function values and gradients that are provided by this website to solve the specified class of optimization problems. This approach is very well suited for optimization problems originating from classical machine learning problems.

Papers

For more information and a number of experiments, please see our paper:

Soeren Laue, Mark Blacher, and Joachim Giesen.
Optimization for Classical Machine Learning Problems on the GPU, In AAAI Conference on Artificial Intelligence Advances (AAAI), 2022.

Soeren Laue, Matthias Mitterreiter, and Joachim Giesen.
GENO -- GENeric Optimization for Classical Machine Learning, In Advances in Neural Information Processing Systems (NeurIPS), 2019.