Documentation

You can enter your optimization problem into the code editor. If you press the 'Download Solver'-button, Python code is generated automatically that can solve this class of optimization problems. Some random sample data is generated as well, so you can run this code right away. The generated code provides functions that can compute the function values and derivatives of the objective function and the constraints. These functions are either fed into the SLSQP solver from the SciPy package or into the GENO solver. We suggest using the GENO solver as it is much faster and scales to problems involving thousands of variables and constraints. The GENO solver can be installed using the 'pip install genosolver' command.

The GENO Modeling Language

Consider for instance the following optimization problem:

\[ \begin{array}{rl} \displaystyle \min_{x} & c^\top \cdot x \\ \mbox{st} & A\cdot x \leq b \\ & x \geq 0, \end{array} \]

where \(A, b, c\) are given parameters and \(x\) is the optimization variable. This problem can be written in GENO as:

parameters matrix A vector b, c variables vector x min c'*x st A*x <= b x >= 0

Details

A GENO specification has four blocks:

  1. Declaration of the problem parameters that can be of type matrix, vector, or scalar,
  2. declaration of the optimization variables that also can be of type matrix, vector, or scalar,
  3. specification of the objective function in a MATLAB-like syntax, and finally
  4. specification of the constraints, also in a MATLAB-like syntax that supports the following operators and functions: +, -, *, /, .*, ./, ^, .^, log, exp, sin, cos, tanh, abs, norm1, norm2, sum, tr, det, inv.

Input matrices can be specified as  symmetric, psd (positive semidefinite), or  sparse. The constraints section can also be omitted.

Examples

The following examples illustrate the GENO modeling language:

Least Squares Regression
parameters matrix A vector b variables vector x min norm2(A*x-b)^2
Non-negative Robust Regression
parameters matrix A vector b variables vector x min norm1(A*x-b) st x >= 0
Symmtric, Non-negative Matrix Factorization
parameters Matrix X symmetric variables Matrix U min norm2(X - U*U')^2 st U >= 0
Quadratic Function over the Unit Simplex
parameters matrix Q psd vector c variables vector x min 0.5*x'*Q*x + c'*x st sum(x) == 1 x >= 0

The GENO Solver

The GENO solver combines an Augmented Lagrangian approach with a limited memory quasi-Newton method (L-BFGS-B) that can handle also bound constraints on the variables. Quasi-Newton methods are very efficient for problems involving thousands of optimization variables. The GENO solver is then instantiated by the automatically generated methods for computing function values and gradients that are provided by this website to solve the specified class of optimization problems. This approach is very well suited for optimization problems originating from classical machine learning problems.

Paper

For more information and a number of experiments, please see our paper:

Soeren Laue, Matthias Mitterreiter, and Joachim Giesen.
GENO -- GENeric Optimization for Classical Machine Learning, In Advances in Neural Information Processing Systems (NeurIPS), 2019.