Future Solver Interface Changes

Note

The new solver interfaces are still under active development. They are included in the releases as development previews. Please be aware that APIs and functionality may change with no notice.

We welcome any feedback and ideas as we develop this capability. Please post feedback on Issue 1030.

Pyomo offers interfaces into multiple solvers, both commercial and open source. To support better capabilities for solver interfaces, the Pyomo team is actively redesigning the existing interfaces to make them more maintainable and intuitive for use. A preview of the redesigned interfaces can be found in pyomo.contrib.solver.

New Interface Usage

The new interfaces are not completely backwards compatible with the existing Pyomo solver interfaces. However, to aid in testing and evaluation, we are distributing versions of the new solver interfaces that are compatible with the existing (“legacy”) solver interface. These “legacy” interfaces are registered with the current SolverFactory using slightly different names (to avoid conflicts with existing interfaces).

Table 7 Available Redesigned Solvers and Names Registered in the SolverFactories

Solver

Name registered in the
pyomo.contrib.solver.common.factory.SolverFactory

Name registered in the
pyomo.opt.base.solvers.LegacySolverFactory

Ipopt

ipopt

ipopt_v2

Gurobi (persistent)

gurobi_persistent

gurobi_persistent_v2

Gurobi (direct)

gurobi_direct

gurobi_direct_v2

HiGHS

highs

highs

KNITRO

knitro_direct

knitro_direct

GAMS

gams

gams_v2

Using the new interfaces through the legacy interface

Here we use the new interface as exposed through the existing (legacy) solver factory and solver interface wrapper. This provides an API that is compatible with the existing (legacy) Pyomo solver interface and can be used with other Pyomo tools / capabilities.

import pyomo.environ as pyo

model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)

def rosenbrock(model):
    return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2

model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)

status = pyo.SolverFactory('ipopt_v2').solve(model)
pyo.assert_optimal_termination(status)
model.pprint()

In keeping with our commitment to backwards compatibility, both the legacy and future methods of specifying solver options are supported:

import pyomo.environ as pyo

model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)

def rosenbrock(model):
    return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2

model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)

# Backwards compatible
status = pyo.SolverFactory('ipopt_v2').solve(model, options={'max_iter' : 6})
# Forwards compatible
status = pyo.SolverFactory('ipopt_v2').solve(model, solver_options={'max_iter' : 6})
model.pprint()

Using the new interfaces directly

Here we use the new interface by importing it directly:

# Direct import
import pyomo.environ as pyo
from pyomo.contrib.solver.solvers.ipopt import Ipopt

model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)

def rosenbrock(model):
    return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2

model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)

opt = Ipopt()
status = opt.solve(model)
pyo.assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()

Using the new interfaces through the “new” SolverFactory

Here we use the new interface by retrieving it from the new SolverFactory:

# Import through new SolverFactory
import pyomo.environ as pyo
from pyomo.contrib.solver.common.factory import SolverFactory

model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)

def rosenbrock(model):
    return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2

model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)

opt = SolverFactory('ipopt')
status = opt.solve(model)
pyo.assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()

Switching all of Pyomo to use the new interfaces

We also provide a mechanism to get a “preview” of the future where we replace the existing (legacy) SolverFactory and utilities with the new (development) version (see Accessing preview features):

# Change default SolverFactory version
import pyomo.environ as pyo
from pyomo.__future__ import solver_factory_v3

model = pyo.ConcreteModel()
model.x = pyo.Var(initialize=1.5)
model.y = pyo.Var(initialize=1.5)

def rosenbrock(model):
    return (1.0 - model.x) ** 2 + 100.0 * (model.y - model.x**2) ** 2

model.obj = pyo.Objective(rule=rosenbrock, sense=pyo.minimize)

status = pyo.SolverFactory('ipopt').solve(model)
pyo.assert_optimal_termination(status)
# Displays important results information; only available through the new interfaces
status.display()
model.pprint()

Linear Presolve and Scaling

The new interface allows access to new capabilities in the various problem writers, including the linear presolve and scaling options recently incorporated into the redesigned NL writer. For example, you can control the NL writer in the new ipopt interface through the solver’s writer_config configuration option (see the Ipopt interface documentation).

from pyomo.contrib.solver.solvers.ipopt import Ipopt
opt = Ipopt()
opt.config.writer_config.display()
show_section_timing: false
skip_trivial_constraints: true
file_determinism: FileDeterminism.ORDERED
symbolic_solver_labels: false
scale_model: true
export_nonlinear_variables: None
row_order: None
column_order: None
export_defined_variables: true
linear_presolve: true

Note that, by default, both linear_presolve and scale_model are enabled. Users can manipulate linear_presolve and scale_model to their preferred states by changing their values.

>>> opt.config.writer_config.linear_presolve = False

Interface Implementation

All new interfaces should be built upon one of two classes (currently): SolverBase or PersistentSolverBase.

All solvers should have the following:

class pyomo.contrib.solver.common.base.SolverBase(**kwds)[source]

The base class for “new-style” Pyomo solver interfaces.

This base class defines the methods all derived solvers are expected to implement:

Class Configuration

All derived concrete implementations of this class must define a class attribute CONFIG containing the ConfigDict that specifies the solver’s configuration options. By convention, the CONFIG should derive from (or implement a superset of the options from) one of the following:

classmethod api_version()[source]

Return the public API supported by this interface.

Returns:

A solver API enum object

Return type:

SolverAPIVersion

available() Availability[source]

Test if the solver is available on this system.

Nominally, this will return True if the solver interface is valid and can be used to solve problems and False if it cannot. Note that for licensed solvers there are a number of “levels” of available: depending on the license, the solver may be available with limitations on problem size or runtime (e.g., ‘demo’ vs. ‘community’ vs. ‘full’). In these cases, the solver may return a subclass of enum.IntEnum, with members that resolve to True if the solver is available (possibly with limitations). The Enum may also have multiple members that all resolve to False indicating the reason why the interface is not available (not found, bad license, unsupported version, etc).

Returns:

available – An enum that indicates “how available” the solver is. Note that the enum can be cast to bool, which will be True if the solver is runnable at all and False otherwise.

Return type:

Availability

is_persistent() bool[source]

True if this supports a persistent interface to the solver.

Returns:

is_persistent – True if the solver is a persistent solver.

Return type:

bool

solve(model: BlockData, **kwargs) Results[source]

Solve a Pyomo model.

Parameters:
  • model (BlockData) – The Pyomo model to be solved

  • **kwargs – Additional keyword arguments (including solver_options - passthrough options; delivered directly to the solver (with no validation))

Returns:

results – A results object

Return type:

Results

version() Tuple[source]

Return the solver version found on the system.

Returns:

version – A tuple representing the version

Return type:

tuple

config

Instance configuration; see CONFIG documentation on derived class

Persistent solvers include additional members as well as other configuration options:

class pyomo.contrib.solver.common.base.PersistentSolverBase(**kwds)[source]

Bases: SolverBase

Base class upon which persistent solvers can be built. This inherits the methods from the solver base class and adds those methods that are necessary for persistent solvers.

Example usage can be seen in the Gurobi interface.

add_block(block: BlockData)[source]

Add a block to the model.

add_constraints(cons: List[ConstraintData])[source]

Add constraints to the model.

is_persistent() bool[source]
Returns:

is_persistent – True if the solver is a persistent solver.

Return type:

bool

remove_block(block: BlockData)[source]

Remove a block from the model.

remove_constraints(cons: List[ConstraintData])[source]

Remove constraints from the model.

set_instance(model: BlockData)[source]

Set an instance of the model.

set_objective(obj: ObjectiveData)[source]

Set current objective for the model.

solve(model: BlockData, **kwargs) Results[source]

Solve a Pyomo model.

Parameters:
  • model (BlockData) – The Pyomo model to be solved

  • **kwargs – Additional keyword arguments (including solver_options - passthrough options; delivered directly to the solver (with no validation))

Returns:

results – A results object

Return type:

Results

update_parameters()[source]

Update parameters on the model.

update_variables(variables: List[VarData])[source]

Update variables on the model.

Results

Every solver, at the end of a solve call, will return a Results object. This object is a pyomo.common.config.ConfigDict, which can be manipulated similar to a standard dict in Python.

class pyomo.contrib.solver.common.results.Results(description=None, doc=None, implicit=False, implicit_domain=None, visibility=0)[source]

Bases: ConfigDict

Base class for all solver results

display(content_filter=None, indent_spacing=2, ostream=None, visibility=0)[source]

Print the current Config value, in YAML format.

The current values stored in this Config object are output to ostream (or sys.stdout if ostream is None). If visibility is not None, then only items with visibility less than or equal to visibility will be output. Output can be further filtered by providing a content_filter.

extra_info: ConfigDict
incumbent_objective: float | None
objective_bound: float | None
solution_status: SolutionStatus
solver_config: ConfigDict
solver_log: str
solver_name: str | None
solver_version: Tuple[int, ...] | None
termination_condition: TerminationCondition
timing_info: ConfigDict

The new interface has condensed SolverStatus, TerminationCondition, and SolutionStatus into TerminationCondition and SolutionStatus to reduce complexity. As a result, several legacy SolutionStatus values are no longer achievable. These are detailed in the table below.

Table 8 Mapping from unachievable SolutionStatus to future statuses

Legacy SolutionStatus

TerminationCondition

SolutionStatus

other

unknown

noSolution

unsure

unknown

noSolution

locallyOptimal

convergenceCriteriaSatisfied

optimal

globallyOptimal

convergenceCriteriaSatisfied

optimal

bestSoFar

convergenceCriteriaSatisfied

feasible

Termination Conditions

Pyomo offers a standard set of termination conditions to map to solver returns. The intent of TerminationCondition is to notify the user of why the solver exited. The user is expected to inspect the Results object or any returned solver messages or logs for more information.

class pyomo.contrib.solver.common.results.TerminationCondition(*values)[source]

Bases: Enum

An Enum that enumerates all possible exit statuses for a solver call.

Solution Status

Pyomo offers a standard set of solution statuses to map to solver output. The intent of SolutionStatus is to notify the user of what the solver returned at a high level. The user is expected to inspect the Results object or any returned solver messages or logs for more information.

class pyomo.contrib.solver.common.results.SolutionStatus(*values)[source]

Bases: Enum

An enumeration for interpreting the result of a termination. This describes the designated status by the solver to be loaded back into the model.

Solution

Solutions can be loaded back into a model using a SolutionLoader. A specific loader should be written for each unique case. Several have already been implemented. For example, for ipopt:

class pyomo.contrib.solver.solvers.ipopt.IpoptSolutionLoader(sol_data: ASLSolFileData, nl_info: NLWriterInfo)[source]

Bases: ASLSolFileSolutionLoader

get_duals(cons_to_load: Sequence[ConstraintData] | None = None) dict[ConstraintData, float]

Returns a dictionary mapping constraint to dual value.

Parameters:

cons_to_load (list) – A list of the constraints whose duals should be retrieved. If cons_to_load is None, then the duals for all constraints will be retrieved.

Returns:

duals – Maps constraints to dual values

Return type:

dict

get_primals(vars_to_load: Sequence[VarData] | None = None) Mapping[VarData, float]

Returns a ComponentMap mapping variable to var value.

Parameters:

vars_to_load (list) – A list of the variables whose solution value should be retrieved. If vars_to_load is None, then the values for all variables will be retrieved.

Returns:

primals – Maps variables to solution values

Return type:

ComponentMap

get_reduced_costs(vars_to_load: Sequence[VarData] | None = None) Mapping[VarData, float][source]

Returns a ComponentMap mapping variable to reduced cost.

Parameters:

vars_to_load (list) – A list of the variables whose reduced cost should be retrieved. If vars_to_load is None, then the reduced costs for all variables will be loaded.

Returns:

reduced_costs – Maps variables to reduced costs

Return type:

ComponentMap

load_vars(vars_to_load: Sequence[VarData] | None = None) None

Load the solution of the primal variables into the value attribute of the variables.

Parameters:

vars_to_load (list) – The minimum set of variables whose solution should be loaded. If vars_to_load is None, then the solution to all primal variables will be loaded. Even if vars_to_load is specified, the values of other variables may also be loaded depending on the interface.

Dual Sign Convention

For all future solver interfaces, Pyomo adopts the following sign convention. Given the problem

\[\begin{split}\begin{aligned} \min\quad & f(x) \\ \text{s.t.}\quad & c_i(x) = 0 \quad \forall i \in \mathcal{E} \\ & g_i(x) \le 0 \quad \forall i \in \mathcal{U} \\ & h_i(x) \ge 0 \quad \forall i \in \mathcal{L} \end{aligned}\end{split}\]

We define the Lagrangian as

\[\begin{aligned} L(x, \lambda, \nu, \delta) &= f(x) - \sum_{i \in \mathcal{E}} \lambda_i\,c_i(x) - \sum_{i \in \mathcal{U}} \nu_i\,g_i(x) - \sum_{i \in \mathcal{L}} \delta_i\,h_i(x) \end{aligned}\]

Then, the KKT conditions are [NW99]

\[\begin{split}\begin{aligned} \nabla_x L(x, \lambda, \nu, \delta) &= 0 \\ c(x) &= 0 \\ g(x) &\le 0 \\ h(x) &\ge 0 \\ \nu &\le 0 \\ \delta &\ge 0 \\ \nu_i\,g_i(x) &= 0 \\ \delta_i\,h_i(x) &= 0 \end{aligned}\end{split}\]

Note that this sign convention is based on the (lower, body, upper) representation of constraints rather than the expression provided by a user. Users can specify constraints with variables on both the left- and right-hand sides of equalities and inequalities. However, the (lower, body, upper) representation ensures that all variables appear in the body, matching the form of the problem above.

For maximization problems of the form

\[\begin{split}\begin{aligned} \max\quad & f(x) \\ \text{s.t.}\quad & c_i(x) = 0 \quad \forall i \in \mathcal{E} \\ & g_i(x) \le 0 \quad \forall i \in \mathcal{U} \\ & h_i(x) \ge 0 \quad \forall i \in \mathcal{L} \end{aligned}\end{split}\]

we define the Lagrangian to be the same as above:

\[\begin{aligned} L(x, \lambda, \nu, \delta) &= f(x) - \sum_{i \in \mathcal{E}} \lambda_i\,c_i(x) - \sum_{i \in \mathcal{U}} \nu_i\,g_i(x) - \sum_{i \in \mathcal{L}} \delta_i\,h_i(x) \end{aligned}\]

As a result, the signs of the duals change. The KKT conditions are

\[\begin{split}\begin{aligned} \nabla_x L(x, \lambda, \nu, \delta) &= 0 \\ c(x) &= 0 \\ g(x) &\le 0 \\ h(x) &\ge 0 \\ \nu &\ge 0 \\ \delta &\le 0 \\ \nu_i\,g_i(x) &= 0 \\ \delta_i\,h_i(x) &= 0 \end{aligned}\end{split}\]

Pyomo also supports “range constraints” which are inequalities with both upper and lower bounds, where the bounds are not equal. For example,

\[-1 \leq x + y \leq 1\]

These are handled very similarly to variable bounds in terms of dual sign conventions. For these, at most one “side” of the inequality can be active at a time. If neither side is active, then the dual will be zero. If the dual is nonzero, then the dual corresponds to the side of the constraint that is active. The dual for the other side will be implicitly zero. When accessing duals, the keys are the constraints. As a result, there is only one key for a range constraint, even though it is really two constraints. Therefore, the dual for the inactive side will not be reported explicitly. Again, the sign convention is based on the (lower, body, upper) representation of the constraint. Therefore, the left side of this inequality belongs to \(\mathcal{L}\) and the right side belongs to \(\mathcal{U}\).