Transcript Inventory Routing for Dynamic Waste Collection from Underground Containers Martijn Mes Department of Operational Methods for Production and Logistics University of Twente The Netherlands Monday, November 14,

Inventory Routing for Dynamic
Waste Collection from
Underground Containers
Martijn Mes
Department of Operational Methods for Production and Logistics
University of Twente
The Netherlands
Monday, November 14, 2011
INFORMS Annual Meeting 2011, Charlotte, NC
OUTLINE
 Case introduction
 The company
 The underground container project
 Dynamic collection policies




The Inventory Routing Problem
Heuristic approach
Optimization approach
Conclusions
INFORMS Annual Meeting 2011
2/41
THE COMPANY
 Twente Milieu: a waste collection company located in the
Netherlands.
 Main activity: collection and processing of waste.
 But also: cleaning of streets and sewers, mowing of verges,
road ice control, and the control of plague animals.
 One of the largest waste collectors in the Netherlands when
it comes to the #households connected to their network.
 Yearly collection of around 225,000,000 kg of waste from a
population of around 400,000 inhabitants.
INFORMS Annual Meeting 2011
3/41
INFORMS Annual Meeting 2011
4/41
TYPE OF CONTAINERS
Mini containers
Block containers
One per household; have to
be put along the side of the
road on pre-defined days.
One for multiple households; mostly located at
apartment buildings; freely accessible.
INFORMS Annual Meeting 2011
5/41
UNDERGROUND CONTAINERS
INFORMS Annual Meeting 2011
6/41
INFORMS Annual Meeting 2011
7/41
ADVANTAGES UNDERGROUND CONTAINERS
 Can be used at all places: apartments, houses, business
parks, within the city centre etc. (≠ mini containers)
 Don’t have to be emptied on pre-defined days (≠ mini
containers)
 Much larger then the block containers (typically 5m3 which is
5 times the volume of a block container)
 Only accessible with a personal card
 Avoids illegal waste deposits (≠ block containers)
 Enables the introduction of ‘Diftar’: charging waste disposal at
different rates per kg depending on the type of garbage
 Less odour nuisance due to solid locking (≠ block containers)
 Contributes to an attractive environment (≠ block containers)
INFORMS Annual Meeting 2011
8/41
INFORMS Annual Meeting 2011
9/41
USING THE UNDERGROUND CONTAINERS
 Between 2009 and 2011, around 700 underground containers have
been installed; 800 new containers will be added soon.
 Containers are equipped with a motion sensor: the number of lid
openings are communicated to Twente Milieu.
 There is a static cyclic schedule that states which containers have to
be emptied on what day. For example: container X has to be emptied
every Tuesday and container Y has to be emptied on Friday once in
the two weeks.
 Every workday, a planning employee assigns trucks and drivers to
the pre-defined containers. On Fridays, the planner uses the sensor
information to include some additional urgent containers, thereby
slightly deviating from the static cyclic schedule.
 Why not using this sensor information for the whole selection
process?
INFORMS Annual Meeting 2011
10/41
DYNAMIC WASTE COLLECTION
 Dynamic planning methodology: each day, select the
containers to be emptied based on their estimated fill levels
(using sensor information).
 Research objective:
To asses in what way and up to what degree a dynamic
planning methodology can be used by Twente Milieu to
increase efficiency in the emptying process of underground
containers in terms of logistical costs, customer satisfaction,
and CO2 emissions.
INFORMS Annual Meeting 2011
11/41
INVENTORY ROUTING PROBLEM
 In the literature, our problem is known as a Inventory Routing
Problem (IRP) which combines:
 The vehicle routing problem (VRP)
 Inventory Management \ Vendor Managed Inventory (VMI)
 Trade-off decisions:
 When to deliver a customer?
 How much to deliver a customer?
 Which delivery routes to use?
 The current cyclic planning approach relates to the Periodic
Vehicle Routing Problem (PVRP):
 A multi-period VRP where customers have to be visited a
given number of times within a given planning horizon
(decision on visit combinations and routes).
INFORMS Annual Meeting 2011
12/41
ILLUSTRATION OF THE IRP
 Basic question for IRPs: which customers to serve today
and how to route our trucks?
Enough empty
space left
Depot
Empty space
needs to be
delivered soon
Parking
INFORMS Annual Meeting 2011
13/41
SOLUTION METHODOLOGIES FOR IRPs
 ILP\SDP\MDP\Heuristics:











Federgruen and Zipkin (1984), A Combined Vehicle Routing and Inventory
Allocation Problem.
Campbell et al. (1997), The Inventory Routing Problem.
Bard et al. (1998), A Decomposition Approach to the Inventory Routing
Problem with Satellite Facilities.
Chan et al. (1998), Probabilistic Analyses and Practical Algorithms for
Inventory-Routing Models.
Berman et al. (2001), Deliveries in an inventory/routing problem using
stochastic dynamic programming.
Kleywegt et al. (2002), The Stochastic inventory routing problem with direct
deliveries.
Adelman (2004), A Price-Directed Approach to Stochastic Inventory/Routing.
Campbell et al. (2004), A decomposition approach for the inventory-routing
problem.
Kleywegt et al. (2004), Dynamic programming approximations for a
stochastic inventory routing problem.
Archetti et al. (2007), A branch-and-cut algorithm for a vendor-managed
inventory-routing problem.
Bard et al. (2009), The integrated production–inventory–distribution–routing
problem.
INFORMS Annual Meeting 2011
14/41
OUR SOLUTION METHODOLOGY
 Some characteristics of our problem:
 Multi-vehicle: up to 7 trucks.
 Multi-depot: 2 parking areas and 1 waste processing center.
 Large-scale: expanding to 1500 customers (containers),
which requires > 300 visits per day.
 Long planning horizon: a short-term planning approach will
postpone deliveries to the next period.
 Dynamic environment: stochastic travel times and waste
disposals → we have to be able to do replanning.
 Changing environment: seasonal patters and special days.
 To cope with these characteristics, we use a fast heuristic.
 To anticipate changes in waste disposal, we equip our
heuristic with a number of tunable parameters and optimize
over these parameters.
INFORMS Annual Meeting 2011
15/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
MayGo
MustGo
Depot
Parking
INFORMS Annual Meeting 2011
16/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Seed
Depot
Parking
INFORMS Annual Meeting 2011
17/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
INFORMS Annual Meeting 2011
18/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
INFORMS Annual Meeting 2011
19/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
INFORMS Annual Meeting 2011
20/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
INFORMS Annual Meeting 2011
21/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
INFORMS Annual Meeting 2011
22/41
BASIC IDEA OF THE HEURISTIC
 Create initial routes based on MustGo’s (seed customers and
workload balancing) and extend these routes with MayGo’s.
Depot
Parking
Extended with MayGo’s
INFORMS Annual Meeting 2011
23/41
1. Start
2. Initialize schedules
3. Initial computations
4. Plan seeds
5. Balance workload
6. Plan MustGo’s
7. Plan MayGo’s
ALGORITHM OUTLINE
1. Initial planning in the morning and replanning during the day.
2. Empty schedules in a non-preemtive way and keep them feasible.
3. Estimate the days left; MustGo’s (days left < MustGoDay); optional
workload balancing (to avoid peaks on Mondays and Fridays);
trucks to use; lower bound on the number of routes to use.
4. One seed per truck to (i) spread trucks across the area, (ii) realize
container insertions both close and far from the depot, and (iii)
balance the workload per route to anticipate later MayGo
insertions; seeds based on largest minimum distance from the
depot and other seeds; Assign routes to trucks.
5. Optionally, assign MustGo’s to trucks or routes in a balanced way
(in anticipation of MayGo insertions).
6. Plan all remaining MustGo’s based on cheapest insertion costs.
7. Play MayGo’s: see next sheet.
8. Execute planning and perform replanning when needed.
8. End
INFORMS Annual Meeting 2011
24/41
ADDING MAYGO CONTAINERS
 MayGo’s: days left < MustGoDay+MayGoDay.
 Planning extremes:
 Wait first: MayGoDay=0
 Drive first: MayGoDay=∞
 The best option would be somewhere in between.
 Selection of MayGo’s depend on the additional travel time
(insertion costs) as well as the inventory (volume garbage).
 Options:
 Ratio insertion costs / inventory.
 Relative improvement of this ratio compared to a smoothed
historical ratio. A large positive value indicates an
opportunity we should take.
 Use (optional) limit on the number of MayGo’s.
INFORMS Annual Meeting 2011
25/41
WILL IT WORK? A SIMULATION STUDY
 Benchmark the current way of working and gain insight in
the performance of our heuristic
INFORMS Annual Meeting 2011
26/41
NUMERICAL RESULTS
 Based on current deposit volumes and truck capacity,
savings of 14.6% can be achieved, which consists of 40%
reduction of penalty costs and 18% less travel distance.
 Savings increase with decreasing truck capacities.
40.0%
StaticS
Dynamic
DynamicS
Savings wrt Static
30.0%
20.0%
10.0%
0.0%
0.5
0.7
0.9
1.1
Varying max emptyings per day
1.3
1.5
INFORMS Annual Meeting 2011
27/41
OBSERVATIONS
 Performance heavily depends on the parameter settings:
 MustGoDay
 MayGoDay
 MaxPerDay (to limit MayGo’s)
 NrTrucks
 Slack capacity in trucks (to avoid replanning)
 Etc.
 Moreover, the “right settings” for these parameters heavily
depend on the day of the week.
 We could learn these parameters
 Through experimentation in practice (online learning)
 Through simulation experiments (offline learning)
INFORMS Annual Meeting 2011
28/41
STOCHASTIC SEARCH
 Where is the min\max of some multi-dimensional function
when the surface is measured with noise?
 In our case: at least a 10 dimensional function (using only the
parameters MustGoDay and MayGoDay for 5 workdays).
INFORMS Annual Meeting 2011
29/41
SIMULATION OPTIMIZATION
 The optimization problem:
Vector or parameters to
be adjusted (MustGoDay,
MayGoDay, NrTrucks,
etc., for all working days)
min f  x 
x X
Set of all parameter combinations
•
•
•
Unknown function (no
closed-form formulation)
We can measure it
Measurement will not be
exact (we measure with
noise y=f(x)+ε)
 Simulation optimization:
 The measurements follow from a simulation run.
 Hence, these measurements are expensive.
 Hence, we aim to reduce the required number of
measurements.
 Approaches: Heuristic methods (genetic algorithms, simulated annealing,
tabu search etc.); Response Surface Methods (RSM); Stochastic
Approximation (SA) methods; Bayesian Global Optimization (BGO).
INFORMS Annual Meeting 2011
30/41
BAYESIAN GLOBAL OPTIMIZATION
 Bayesian optimization involves three stages:
1.
Designing the prior distribution (belief about f)
2.
Updating this distribution using Bayes' rule
3.
Deciding what values to sample next
 Often, the belief about f conforms to a Gaussian process.
 A Gaussian process is a collection of random variables {yx1,
yx2,…} for which any finite subset has a joint multivariate
Gaussian (Normal) distribution:
yx ~ N x , k x, x
Measurements
Mean
Kernel function (covariance
between two variables)
INFORMS Annual Meeting 2011
31/41
MORE INFORMATION ON BGO
 Daniel Lizotte (2008)
Practical Bayesian Optimization, PhD Thesis.
 Eric Brochu, Mike Cora and Nando de Freitas (2009)
A Tutorial on Bayesian Optimization of Expensive Cost
Functions, with Application to Active User Modeling and
Hierarchical Reinforcement Learning.
 INFORMS Tutorial by Peter Frazier today from 16:30-18:00
Bayesian Methods for Global and Simulation Optimization.
INFORMS Annual Meeting 2011
32/41
OPTIMIZATION POLICIES WE CONSIDER
 Sequential Kriging Optimization (SKO) by Huang et al.
(2006) which is an extension of Efficient global optimization
(EGO) by Jones et al. (1998) for noisy measurements.
EGO: new points to be measured are selected based on
“expected improvement” which strikes a balance between
exploitation and exploration.
 Knowledge Gradient for Correlated Beliefs (KGCB) by
Frazier et al. (2009). KG: best we can do given we if there is
only one measurement left to make.
 Hierarchical Knowledge Gradient (HKG) by Mes et al.
(2011). HKG: hierarchical aggregation technique that uses
the common features shared by alternatives to learn about
many alternatives from even a single measurement.
INFORMS Annual Meeting 2011
33/41
ILLUSTRATION OF EGO: N=2
Source: Brochu et al. (2009)
INFORMS Annual Meeting 2011
34/41
ILLUSTRATION OF EGO: N=3
Source: Brochu et al. (2009)
INFORMS Annual Meeting 2011
35/41
ILLUSTRATION OF EGO: N=4
Source: Brochu et al. (2009)
INFORMS Annual Meeting 2011
36/41
ILLUSTRATION OF EGO: N=5
Source: Brochu et al. (2009)
INFORMS Annual Meeting 2011
37/41
ILLUSTRATION OF HKG [EXCEL DEMO]
INFORMS Annual Meeting 2011
38/41
APPLICABILITY OF THESE POLICIES
INFORMS Annual Meeting 2011
39/41
EXPERIMENTS WITH SKO
 Experiment 1: 378 containers with 3 trucks:
Mon
Tue
Wed
Thu
Fri
MustGoDay
4.0
0.0
0.0
1.2
0.0
MayGoDay
4.0
X
X
3.5
X
with a maximum of 113 emptying's per day.
 Experiment 2: 700 containers, 50% higher deposit volumes
and 2 trucks:
Mon
Tue
Wed
Thu
Fri
MustGoDay
1.0
1.1
1.5
2.7
2.1
MayGoDay
0.0
0.0
4.0
4.0
4.0
with a maximum of 672 emptying’s per day.
 Results are counterintuitive at first sight. Still, they result in
additional savings of around 10%.
INFORMS Annual Meeting 2011
40/41
CONCLUSIONS
 We proposed a fast heuristic suitable for Inventory Routing
Problems involving a large number of customers.
 Application of this heuristic to the waste collection problem
is expected to result in a reduction of 18% in travel costs
and 40% in penalty costs (due to waste overflow).
 An optimization approach is preferred to anticipate changes
in waste disposals. To enable this, we equipped our
heuristic with several tunable parameters.
 To optimize over these parameters we used techniques
from Simulation Optimization and Bayesian Global
Optimization (SKO, KGCB, HKG).
 For our waste collection problem, this will result in additional
savings of 10% in total costs (travel costs and penalty
costs).
INFORMS Annual Meeting 2011
41/41
QUESTIONS?
Martijn Mes
Assistant professor
University of Twente
School of Management and Governance
Operational Methods for Production and Logistics
The Netherlands
Contact
Phone: +31-534894062
Email: [email protected]
Web: http://www.utwente.nl/mb/ompl/staff/Mes/