Speed up `-SumExpr`, `-ProdExpr` and `-Constant` by Zeroto521 · Pull Request #1179 · scipopt/PySCIPOpt

Zeroto521 · 2026-01-29T05:16:44Z

Use a C-level to speed up SumExpr.__neg__, ProdExpr.__neg__, and Constant.__neg__.
SumExpr is 2.1x faster than before, and ProdExpr is 4.9x faster than before.

Optimized before
- -SumExpr time: 1.38s
- -ProdExpr time: 1.39s
Optimized after
- -SumExpr time: 0.64s
- -ProdExpr time: 0.28s

from timeit import timeit

from pyscipopt import Model, sin


m = Model()
x = m.addVar("x")
y = m.addVar("y")

sumexpr = sin(x) + x + y + 1
prodexpr = sumexpr * -42
print(f"SumExpr: {sumexpr}")
# SumExpr: sum(1.0,sin(sum(0.0,prod(1.0,x))),prod(1.0,x),prod(1.0,y))
print(f"ProdExpr: {prodexpr}")
# ProdExpr: prod(-42.0,sum(1.0,sin(sum(0.0,prod(1.0,x))),prod(1.0,x),prod(1.0,y)))

n = 1000000
print(f"-SumExpr time: {timeit(lambda: -sumexpr, number=n):.2f}s")
print(f"-ProdExpr time: {timeit(lambda: -prodexpr, number=n):.2f}s")

Refactors the __neg__ method in the Expr class to use Cython's PyDict_Next and PyDict_SetItem for more efficient negation of terms, replacing the previous Python dict comprehension.

Introduces a copy method to GenExpr for duplicating expression objects, with support for deep or shallow copying. Also implements the __neg__ method for ProdExpr to allow negation of product expressions by negating their constant term.

Added explicit return type annotations to the __neg__ methods in Expr and ProdExpr classes, and updated the corresponding type hints in scip.pyi. This improves type checking and code clarity.

Refactors SumExpr to use cpython.array for storing coefficients instead of Python lists, improving performance and memory efficiency. Adds a __neg__ method for SumExpr to efficiently negate coefficients and the constant term. Updates the copy method to properly clone arrays when copying SumExpr instances.

Introduces the test_neg function to verify correct behavior when negating ProdExpr and SumExpr objects in the expression API. Ensures that negated expressions have the expected types, string representations, and coefficients.

Added an entry noting the speedup of `Expr.__neg__`, `ProdExpr.__neg__`, and `Constant.__neg__` using the C-level API.

The @disjoint_base decorator was removed from the UnaryExpr class in the type stub, possibly to correct or update the class hierarchy or decorator usage.

Copilot

Pull request overview

This PR aims to speed up expression negation by moving __neg__ implementations to faster C-level loops for core expression types.

Changes:

Optimized Expr.__neg__ by iterating the underlying term dictionary using CPython C-API helpers.
Added specialized __neg__ implementations for SumExpr and ProdExpr (including new GenExpr.copy helper).
Added a new unit test covering negation behavior for ProdExpr and SumExpr, and refined type stubs for __neg__.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File	Description
`tests/test_expr.py`	Adds a `test_neg` test validating negation results/types for product and sum expressions.
`src/pyscipopt/scip.pyi`	Tightens typing for `__neg__` on `Expr` and `GenExpr`.
`src/pyscipopt/expr.pxi`	Implements faster negation paths and introduces `GenExpr.copy`; changes `SumExpr.coefs` storage to `array('d')`.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Changed the type of 'coefs' from list to memoryview (double[:]) in SumExpr._evaluate for more efficient access during evaluation.

Refactors the negation method in SumExpr to correctly create a new instance, negate the constant, copy children, and set the operator. This ensures proper behavior when negating sum expressions.

Implemented the __neg__ method for the Constant class, allowing unary negation of constant expressions.

Imported Constant from pyscipopt.scip and added an assertion to test the string representation of the negated Constant expression in test_neg.

Added assertions to test_neg to verify correct negation and string representation of expressions involving powers. This enhances test coverage for expression negation logic.

Clarified that `SumExpr.__neg__` is also sped up via the C-level API, in addition to other negation methods.

Zeroto521 · 2026-01-30T02:56:35Z

    def __init__(self):
        self.constant = 0.0
-        self.coefs = []
+        self.coefs = array("d")


@Joao-Dionisio Could you have a loook about this problem? SumExpr.coefs has a bug, so this CI failed.
If SumExpr.coefs is not [1, 1], the result from expr_to_array to addCons won't be right.

All GenExpr will use the expr_to_array function to add to SCIP. But expr_to_array doesn't have any code to handle SumExpr.coefs. SumExpr.coefs doesn't work at all.

PySCIPOpt/src/pyscipopt/expr.pxi

Lines 829 to 849 in bc24757

def expr_to_array(expr, nodes):

"""adds expression to array"""

op = expr._op

if op == Operator.const: # FIXME: constant expr should also have children!

nodes.append(tuple([op, [expr.number]]))

elif op != Operator.varidx:

indices = []

nchildren = len(expr.children)

for child in expr.children:

pos = expr_to_array(child, nodes) # position of child in the final array of nodes, 'nodes'

indices.append(pos)

if op == Operator.power:

pos = value_to_array(expr.expo, nodes)

indices.append(pos)

elif (op == Operator.add and expr.constant != 0.0) or (op == Operator.prod and expr.constant != 1.0):

pos = value_to_array(expr.constant, nodes)

indices.append(pos)

nodes.append( tuple( [op, indices] ) )

else: # var

nodes.append( tuple( [op, expr.children] ) )

return len(nodes) - 1

Two solutions to fix this.

Remove SumExpr.coefs. Any coefficient to a SumExpr will be a ProdExpr. And each element coefficient should be 1. We follow current behavior. So, SumExpr.coefs related codes could be removed.

Let expr_to_array support SumExpr.coefs.

SumExpr.coefs has a bug. So the latest CI is still failing.

Hey @Zeroto521 what's the status on this?

SumExpr.coefs has a bug, as I said before. There are two solutions we could fix that. @Joao-Dionisio

Remove SumExpr.coefs. Any coefficient to a SumExpr will be a ProdExpr. And each element coefficient should be 1. We follow current behavior. So, SumExpr.coefs related codes could be removed.

Let expr_to_array support SumExpr.coefs.

I'd rather extend the support of expr_to_array. A suggestion:

In expr_to_array: for Operator.add, include expr.coefs and expr.constant in the node tuple instead of appending the constant

In scip.pxi (the Operator.add branch): read the actual coefs and constant from the node and pass them to SCIPcreateExprSum.

Do you think this might fix things?

I want to drop SumExpr.coefs now.
When I'm working on expr_to_array, I find that SumExpr.coefs can't be printed well. The coefficient should be close to its value. But children and coefs both are list, they are not a single dict. Although we could use zip, it is not a better way for __repr__.

def __repr__(self): return self._op + "(" + str(self.constant) + "," + ",".join(map(lambda child : child.__repr__(), self.children)) + ")"

Replaces usage of cpython.array for storing coefficients in SumExpr with standard Python lists. Simplifies code by removing array-specific imports and clone operations, improving maintainability and compatibility.

Removed the unused GenExpr.copy() method and refactored the __neg__ methods for SumExpr and ProdExpr to avoid using the copy method. This simplifies the code and clarifies object construction during negation.

Applied the @disjoint_base decorator to the UnaryExpr class in scip.pyi to clarify its role in the type hierarchy.

Move and centralize operator overloads for ExprLike: __radd__, __sub__, __rsub__, __rmul__, __neg__ and __richcmp__ are added earlier in the class and the duplicate implementations later in the file were removed. This refactor cleans up the class definition and avoids duplicated method definitions.

Consolidate reflected true-division handling by adding __rtruediv__ to the ExprLike base class and removing duplicate implementations from Expr and GenExpr. The reflected division now uniformly uses buildGenExprObj(other) / self, reducing code duplication and ensuring consistent behavior across expression types.

Document a refactor that moves several dunder methods (__radd__, __sub__, __rsub__, __rmul__, __richcmp__, __neg__, __rtruediv__) into the ExprLike base class to centralize operator behavior. This change only updates CHANGELOG.md to record the API/internal restructuring.

Add the positional-only marker ('/') to several operator method signatures in ExprLike to prevent passing the operand as a keyword and to align with CPython semantics. Affected methods: __radd__, __sub__, __rsub__, __rmul__, and __rtruediv__. This is a signature-level change only and should not alter runtime behavior.

Update src/pyscipopt/scip.pyi: add missing arithmetic/operator dunder declarations (__radd__, __sub__, __rsub__, __rmul__, __rtruediv__, __neg__) to the ExprLike base stub and remove redundant/operator declarations from Expr and GenExpr. This consolidates common operator signatures in the base protocol, reduces duplication, and improves typing/stub consistency.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Delete an extraneous blank line between the end of ExprLike and the @disjoint_base decorator/Expr class in src/pyscipopt/scip.pyi to tidy file formatting. No functional changes.

Copilot

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Replace Incomplete with object for several ExprLike operator stubs (__radd__, __sub__, __rsub__, __rmul__, __rtruediv__, __neg__) to relax/standardize return and operand typing. Also add missing __rtruediv__ stubs to Expr and GenExpr. These changes improve the type stubs for static type checkers and better reflect Python operator semantics.

Add the positional-only marker (/) to the ExprLike.__neg__ signature in src/pyscipopt/expr.pxi. This enforces that the unary negation method cannot be called with keyword arguments and aligns the signature with Python/Cython expectations, avoiding potential warnings or misuse.

Update type stubs in src/pyscipopt/scip.pyi to change return types of several ExprLike dunder operators from object to Incomplete for improved typing precision. Affected methods: __radd__, __sub__, __rsub__, __rmul__, __rtruediv__, and __neg__. This is a type-only change with no runtime behavior modifications.

Add explicit GenExpr return annotations for __rtruediv__ across implementation and typing stub. Updated src/pyscipopt/expr.pxi to annotate ExprLike, Expr and GenExpr __rtruediv__ methods with -> GenExpr, and updated src/pyscipopt/scip.pyi to change the stub return types from object to GenExpr for Expr.__rtruediv__ and GenExpr.__rtruediv__. This improves static typing consistency between the Cython implementation and the Python stubs.

Update negation operator type hints to return Union[Expr, GenExpr]. - src/pyscipopt/expr.pxi: add a Cython return annotation for __neg__. - src/pyscipopt/scip.pyi: change __neg__ from Incomplete to Union[Expr, GenExpr]. These changes improve static typing and IDE support by accurately describing the result of unary negation on expression-like objects.

Update typing in src/pyscipopt/scip.pyi: change ExprLike.__rtruediv__ return type from Incomplete to GenExpr and remove duplicate __rtruediv__ declarations from Expr and GenExpr. This consolidates the right-division signature in the base ExprLike class to avoid conflicting or redundant stub definitions and improve type consistency.

Update the typing stub for ExprLike.__neg__ in src/pyscipopt/scip.pyi to include the positional-only marker (/). This clarifies that __neg__ accepts no keyword arguments and returns Union[Expr, GenExpr]; no runtime behavior changes.

Replace manual object construction and child copying in ProdExpr.__neg__ with self.copy(copy=True) and negate the copied constant. This simplifies the negation implementation, preserves existing attributes (like operator and children) via the copy method, and reduces duplicated code that could lead to inconsistencies.

Zeroto521 · 2026-05-23T04:56:19Z

Let's drop SumExpr.coefs. Details: #1179 (comment)
Or we can discuss SumExpr.coefs later and finish -ProdExpr and -Constant first.

Zeroto521 added 6 commits January 26, 2026 09:06

Optimize Expr negation with Cython dict iteration

28d3adc

Refactors the __neg__ method in the Expr class to use Cython's PyDict_Next and PyDict_SetItem for more efficient negation of terms, replacing the previous Python dict comprehension.

Add return type annotations to __neg__ methods

e4351fa

Added explicit return type annotations to the __neg__ methods in Expr and ProdExpr classes, and updated the corresponding type hints in scip.pyi. This improves type checking and code clarity.

Add tests for negation of expression objects

f55c222

Introduces the test_neg function to verify correct behavior when negating ProdExpr and SumExpr objects in the expression API. Ensures that negated expressions have the expected types, string representations, and coefficients.

Merge remote-tracking branch 'upstream/master' into expr/__neg__

cb349ce

Copilot AI review requested due to automatic review settings January 29, 2026 05:16

Copilot started reviewing on behalf of Zeroto521 January 29, 2026 05:17 View session

Zeroto521 added 2 commits January 29, 2026 13:19

Update changelog with negation speed improvements

5896012

Added an entry noting the speedup of `Expr.__neg__`, `ProdExpr.__neg__`, and `Constant.__neg__` using the C-level API.

Remove @disjoint_base decorator from UnaryExpr

6387cfa

The @disjoint_base decorator was removed from the UnaryExpr class in the type stub, possibly to correct or update the class hierarchy or decorator usage.

Copilot AI reviewed Jan 29, 2026

View reviewed changes

Comment thread src/pyscipopt/expr.pxi Outdated

Comment thread tests/test_expr.py

Comment thread src/pyscipopt/expr.pxi

Merge branch 'master' into expr/__neg__

eac8db9

Zeroto521 mentioned this pull request Jan 29, 2026

Feature: Merge Expr and GenExpr #1074

Open

Zeroto521 added 4 commits January 29, 2026 15:03

Optimize coefs access in SumExpr evaluation

fecba06

Changed the type of 'coefs' from list to memoryview (double[:]) in SumExpr._evaluate for more efficient access during evaluation.

Fix negation logic in SumExpr class

02e32b5

Refactors the negation method in SumExpr to correctly create a new instance, negate the constant, copy children, and set the operator. This ensures proper behavior when negating sum expressions.

Add negation support to Constant expressions

bd280f6

Implemented the __neg__ method for the Constant class, allowing unary negation of constant expressions.

Add test for negation of Constant expression

2e97cc7

Imported Constant from pyscipopt.scip and added an assertion to test the string representation of the negated Constant expression in test_neg.

Zeroto521 changed the title ~~Speed up -Expr and -ProdExpr and -Constant~~ Speed up -Expr, SumExpr, -ProdExpr and -Constant Jan 29, 2026

Zeroto521 changed the title ~~Speed up -Expr, SumExpr, -ProdExpr and -Constant~~ Speed up -Expr, -SumExpr, -ProdExpr and -Constant Jan 29, 2026

Zeroto521 added 2 commits January 29, 2026 18:14

Expand test_neg to cover negation of power expressions

67ce45d

Added assertions to test_neg to verify correct negation and string representation of expressions involving powers. This enhances test coverage for expression negation logic.

Update CHANGELOG for negation speedup details

40945ad

Clarified that `SumExpr.__neg__` is also sped up via the C-level API, in addition to other negation methods.

Zeroto521 commented Jan 30, 2026

View reviewed changes

Zeroto521 added 4 commits January 30, 2026 11:28

Refactor SumExpr to use Python lists for coefficients

ef034c4

Replaces usage of cpython.array for storing coefficients in SumExpr with standard Python lists. Simplifies code by removing array-specific imports and clone operations, improving maintainability and compatibility.

Remove GenExpr.copy

86678e2

Removed the unused GenExpr.copy() method and refactored the __neg__ methods for SumExpr and ProdExpr to avoid using the copy method. This simplifies the code and clarifies object construction during negation.

Add @disjoint_base decorator to UnaryExpr class

394c682

Applied the @disjoint_base decorator to the UnaryExpr class in scip.pyi to clarify its role in the type hierarchy.

Merge branch 'master' into expr/__neg__

d348d48

Joao-Dionisio reviewed Mar 11, 2026

View reviewed changes

Comment thread src/pyscipopt/expr.pxi

Joao-Dionisio reviewed Mar 11, 2026

View reviewed changes

Comment thread src/pyscipopt/expr.pxi Outdated

Zeroto521 mentioned this pull request Mar 12, 2026

Speed up constant * Expr #1185

Merged

Merge remote-tracking branch 'upstream/master' into expr/__neg__

b881b12

Zeroto521 and others added 9 commits April 6, 2026 20:01

Update CHANGELOG.md

9d2d251

Update CHANGELOG.md

b814b40

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Merge branch 'ExprLike' of github.com:Zeroto521/PySCIPOpt into ExprLike

27ff00f

style: format scip.pyi with ruff

c7f6715

Delete an extraneous blank line between the end of ExprLike and the @disjoint_base decorator/Expr class in src/pyscipopt/scip.pyi to tidy file formatting. No functional changes.

Zeroto521 mentioned this pull request Apr 8, 2026

Speed up Expr __add__ and __iadd__ #1205

Merged

Joao-Dionisio requested a review from Copilot April 22, 2026 13:18

Copilot started reviewing on behalf of Joao-Dionisio April 22, 2026 13:18 View session

Copilot AI reviewed Apr 22, 2026

View reviewed changes

Comment thread src/pyscipopt/expr.pxi

Comment thread src/pyscipopt/expr.pxi

Zeroto521 requested a review from Joao-Dionisio April 24, 2026 02:14

Zeroto521 and others added 15 commits April 24, 2026 10:59

Merge branch 'scipopt:master' into ExprLike

483528a

Merge branch 'master' into ExprLike

d88206c

Update CHANGELOG.md

8e8bf59

Merge branch 'master' into ExprLike

64e6947

Merge branch 'ExprLike' of github.com:Zeroto521/PySCIPOpt into ExprLike

f21d708

Merge branch 'master' into ExprLike

dcd64b9

Mark __neg__ as positional-only in scip.pyi

aa6ae86

Update the typing stub for ExprLike.__neg__ in src/pyscipopt/scip.pyi to include the positional-only marker (/). This clarifies that __neg__ accepts no keyword arguments and returns Union[Expr, GenExpr]; no runtime behavior changes.

Merge branch 'ExprLike' into expr/__neg__

3e3fa95

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Speed up `-SumExpr`, `-ProdExpr` and `-Constant`#1179

Speed up `-SumExpr`, `-ProdExpr` and `-Constant`#1179
Zeroto521 wants to merge 52 commits into
scipopt:masterfrom
Zeroto521:expr/neg

Zeroto521 commented Jan 29, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Zeroto521 Jan 30, 2026 •

edited

Loading

Uh oh!

Zeroto521 Mar 12, 2026

Uh oh!

Joao-Dionisio Mar 30, 2026

Uh oh!

Zeroto521 Mar 31, 2026

Uh oh!

Joao-Dionisio Mar 31, 2026

Uh oh!

Zeroto521 Apr 5, 2026

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Zeroto521 commented May 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	def expr_to_array(expr, nodes):
	"""adds expression to array"""
	op = expr._op
	if op == Operator.const: # FIXME: constant expr should also have children!
	nodes.append(tuple([op, [expr.number]]))
	elif op != Operator.varidx:
	indices = []
	nchildren = len(expr.children)
	for child in expr.children:
	pos = expr_to_array(child, nodes) # position of child in the final array of nodes, 'nodes'
	indices.append(pos)
	if op == Operator.power:
	pos = value_to_array(expr.expo, nodes)
	indices.append(pos)
	elif (op == Operator.add and expr.constant != 0.0) or (op == Operator.prod and expr.constant != 1.0):
	pos = value_to_array(expr.constant, nodes)
	indices.append(pos)
	nodes.append( tuple( [op, indices] ) )
	else: # var
	nodes.append( tuple( [op, expr.children] ) )
	return len(nodes) - 1

Conversation

Zeroto521 commented Jan 29, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Zeroto521 Jan 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Zeroto521 Mar 12, 2026

Choose a reason for hiding this comment

Uh oh!

Joao-Dionisio Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

Zeroto521 Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Joao-Dionisio Mar 31, 2026

Choose a reason for hiding this comment

Uh oh!

Zeroto521 Apr 5, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Zeroto521 commented May 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Zeroto521 commented Jan 29, 2026 •

edited

Loading

Zeroto521 Jan 30, 2026 •

edited

Loading