Skip to main content

Welcome to opt-models!

… a resource for builders of fast, large-scale optimization models in the wild.

Chapters: Released & Coming Soon …

… and more coherently organized in the site menu, above. 😊


The Problem Description
Problem Description Use Case Objectives AMPL HiGHS
SERIES INTRO! # This is the first of eighteen videos over which we build an app for the awesome Rebrickable® community, which finds multiple LEGO® sets buildable concurrently from a user’s part inventory.
Plan the Buildout
App Build Plan Python UI
We sketch a plan for building our optimization application, starting with a bare-bones optimization model and then adding a data connection and simple text-based UI. Later we swap out the AMPL scripting language for Python and the text-based UI for Streamlit, and then containerize the app.
Unbox the Data!
GraphXR EDA Schema
We walk through our LEGO data from and - using GraphXR for exploratory data analysis (EDA) - we translate its schema into an Entity Relationship Diagram, yielding our optimization app’s main entities! (We also meet Legolas Greenleaf, Frankie Poullain and The Darkness …)
Sketch the Model
Variables Objectives Constraints
Step-by-step we ideate model variables and constraints, jotting these down initially as natural language comments in our new AMPL model.mod file.
Connect Model to Data
Data Sources Speed Flat Files AMPL
We establish a connection via an import script (written in the AMPL scripting syntax, later to be replaced by a Python script) to a collection of CSV and flat text data files.
Group SETS
SETS Tuples Speed AMPL
We discuss why grouping sets of single entities into aggregate sets of tuples (aka ‘cartesian products’, ‘cross-joins’) dramatically increases the performance of our optimization app. Also, we discuss the accompanying increase in data memory footprint.
Add a Text-based User Interface
Flat Files UI AMPL
We add a simple but functional, attractive text-based user interface by way of a flat data file formatted for good readability and manual data input.
The Magical Nature of Optimization Programming
Declarative Programming Simplex Problem Object
Optimization programming is an extremely declarative paradigm which is beyond fantastic and also exactly why many newcomers get into trouble.
Write the Model
AMPL Constraints Memory Efficiency
One-by-one we convert our natural language comments representing model vars and constraints into actual model entities. In the process, we add display statements for health-checking the memory size and cardinality of our new model element.
Major Component Overview
Modeling Tools Solvers Data Sources
An overview of the modern optimization toolchain typically deployed in commercial applications: data sources, glue languages, modeling langs/tools, solvers and optional user interfaces.
Shrink the Problem
Speed Set Work Memory Efficiency AMPL
With constraints in place which ensure that selected LEGO® sets are complete, we now implement constraints limiting the search space to sets that the user desires. We then observe that, while functional, a faster-computing way to implement such ‘problem shrinking’ is via set manipulation rather than constraint imposition.
Make on-the-fly model mods
Constraints App Logic Model Assembly at Runtime AMPL
We add functionality to our script that drops unneeded constraints per user-specified options, effectively changing the form of our optimization model on-the-fly. We also discuss extreme approaches to such run-time model construction, in light of the modular nature of optimization model components.
Model + Solver
Algebraic Modeling Languages Solvers Speed AMPL PYOMO GAMS CPLEX Gurobi HiGHS
A close look at large-scale optimization modeling tools and languages, and linear and nonlinear solvers.
Data Sources
Data Sources DataFrames Flat Files
Discussing common data sources for optimization models and their various means of connection to the model.
Tune the Solver!!!
Solver Tuning Speed HiGHS
We explore the solving-time impact of our linear solver’s (HiGHS) various params/options, and use these to tune the solver for significantly faster performance on our optimization problem.
Switch our glue language to Python
Python App Logic AMPL
Having worked thus far in only the AMPL modeling and scripting languages respectively, we jettison the AMPL scripting language and replace it with Python. This transforms our model into a component callable from within Python via the AMPL-Python API.
User Interfaces
UI Flat Files Streamlit
A look at practical as well as fancy means by which humans can interact with optimization models during development and in production.
'Glue' Languages
General Languages Python App Logic
Discussing the use and interchangeability of general languages which ‘glue’ optimization application components together and implement out-of-model app logic.
Add a Streamlit User Interface
Streamlit UI AI-Assisted Coding
Now that our optimization app’s non-model logic script (aka ’the main script’) is in Python, we add a lovely Streamlit UI to replace our text-based UI, created with help from our cheerful AI coding assistant.
Go Dynamic! ...with iterative solving
Dynamic Optimization UI Streamlit AMPL
Our new Streamlit interface makes it easy to imagine and implement an important new feature: the ability to ‘pin’ one or more suggested LEGO® sets, discarding the others, and resolving, so that the user can iteratively arrive at an attractive collection of buildable sets.
The Five Parts of an Optimization Model
Model-Data Separation Constraints Objectives
An unpacking of the five major components of an optimization model: SETS, parameters, Variables, Objective and Constraints.
Dev Creature Comforts
Code & Text Editors Git GraphXR
Extolling the value of tools like Git, fast code/text editors, exploratory data analysis (EDA) tools like GraphXR, whiteboarding tools like Whimsical, containerization, model/data separation and friendly syntax and naming conventions.
Let's Containerize!
Containerization Docker Docker Compose Hardware
We are almost done with our Multi-Build optimization app! Now is a good time to containerize our app so that it can easily be deployed to various environments and even scaled across multiple heterogeneous environments at once.
Spec (rock!) the Hardware ;)
Hardware Speed
Now, let’s dream out loud about some fast hardware platforms to run our app. Physical memory allotment, CPU thread speed, hard disk speed, non-optimization process load, networking and remote database access bandwidth requirements … all impact our hardware specifications.
Mistakes, lies & abandoned features
Problem Description Use Case Mistakes Speed App Build Plan
I reveal some undisclosed turns taken during my initial development of this app. Notably: an abandoned attempt to implement a recursive search for nested sets within sets that would potentially have enabled some cool additional features, and which I might return to later.
RECAP! ...and what happens next
Speed Summary App Handover
A recap of the main points of optimization craft covered in this series, including every major speed-enhancing manuever. I also ponder out loud what interactions with the Rebrickable team might hold for the apps real-world deployment and how to approach this interaction as good citizens.
Descriptive, Predictive, Prescriptive ... analytics
Descriptive Analytics Predictive Analytics Prescriptive Analytics
My perspective on where and why mathematical optimization methods are commonly implemented in the commercial wild, sometimes as the Prescriptive component following earlier Predictive and/or Descriptive components in a modern Data Science pipeline.
Optimization Problem Types
Combinatorial Optimization Summary
A whirlwind tour through the main problem types in the field of Combinatorial Optimization with an emphasis on their various real-world commercial implementations.
Optimization Problem Hardness
Problem Hardness Combinatorial Explosions
Discussing the attributes of an optimization problem which make it harder to solve: combinatorial explosions, a lack of exploitable graph or other structure, etc. Also, a look at the usefulness of concepts like NP-hardness and Big O in addressing optimization problem hardness.
Optimization Methods
Dynamic Optimization Pattern Generation Stochastic Optimization Multi-Objective Optimization Heuristics
A look at various optimization methods such as Stochastic data perturbation, dynamic solving, multi-objective optimization, pattern generation, heuristic and Metaheuristic solvers, and more.
The Myth of the Objective
Serendipity Objectives Use Case
An invitation to consider the application of mathematical optimization in light of the view expressed by Kenneth O. Stanley and Joel Lehman in their book ‘Why Greatness Cannot Be Planned’ (2015). I posit that a richness of possible solutions can in many use cases be more valuable than a closer-to-optimum objective value.
Speeding Opt Apps
Speed Summary Memory Efficiency
A methodical walk across the optimization toolchain and down through the underlying stack/OS in search of performance bottlenecks and their remedies.
Predictions: Opt Tools, Skills and Apps beyond 2024
Predictions Solvers Modeling Tools
My perspective on the markets for optimization products, services and skills in 2024 and beyond. I address developers, students, tool buyers and vendors in light of the current AI explosion.
Optimization ❤️ AI
AI-Assisted Coding Predictions Problem Hardness
A show-and-tell discussion of generative AI’s current value in writing optimization models, data integrations, tests, UIs. Also some discussion about readability of natural vs AML language for humans.
Thank you to...
Gratitude Community Love
Thanks to the folks without whom this would not happen: Seth Godin, Steve Walters, The Black Crowes, Martha Wells and Murderbot, Techno Tim, Christian Lempa, Veronica Explains, Lawrence Systems, Jeff Delaney and, Nuno Coração and Blowfish, the Rebrickable Team, Rebecca and you.
Hey, Community!
Community Love
A call to the optimization community to build valuable resources together: models and integrations, open source solvers, a pro-bono project marketplace, tool-agnostic discussion forum(s).