[Haizea] Developer Docs...

[Borja Sotomayor]

Hi,

> For now, can you please tell me the modules to look for, for lease 
> acceptance and resource allocation process to make any changes that I 
> might want...

Sure! For lease acceptance, the starting point is the rm.py module (the 
"Resource Manager"; this is essentially the root module of Haizea. There 
are some comments in that module that you may find helpful to navigate 
the code).

The RM module can have several "frontends", which is where lease 
requests are taken from. It is the frontends' responsibility to get the 
lease requests and hand them over to the RM, but they make no decision 
regarding whether to accept the request or not (this would be the RM's 
responsibility). When running in simulation, Haizea uses the 
TracefileFrontend class to read from a LWF file (see 
resourcemanager/frontends/tracefile.py and traces/readers.py). When 
running with OpenNebula, Haizea uses the OpenNebulaFrontend, which can 
access the OpenNebula database to obtain lease requests.

The RM will periodically check to see if there are any new requests in 
the frontends. This is done in method process_requests of the 
ResourceManager class (in rm.py). This function has some in-code 
documentation. As you'll see, it basically just collects the pending 
requests and hands them off to the scheduler (method request_lease of 
the Scheduler class in scheduler.py), which will process pending 
requests when the scheduling function is called (method "schedule" of 
the Schedule class)

As you'll see, Haizea currently has an "accept all" policy: all AR 
leases are accepted (as long as there is enough resources to satisfy the 
request) and all best-effort leases are placed in the queue in FCFS 
order (although this order is tweaked using backfilling). The upcoming 
policy engine will make it easy for developers to plug in their own 
policies (e.g., you could write a simple python module that checks if 
the AR lease is requested at least N hours in advance; if not, the lease 
request is rejected).

If you want to modify lease acceptance, I'd say there are two ways of 
going about it:

* If you're going to decide acceptance based on criteria unrelated to 
the current schedule (e.g., what user is requesting the lease, the 
duration of the lease, etc.), I'd say the appropriate place for this 
would be the process_requests method of the ResourceManager class.

* If you're going to decide acceptance based on criteria related to the 
current schedule (e.g., don't accept an AR lease if the current number 
of ARs scheduled uses up more than N% of my resources, etc.), the most 
appropriate place would be the request_lease method of the Scheduler class.


The code for resource allocation is still a bit convoluted, and not as 
well commented as other parts of Haizea (although I hope this will 
gradually improve). The main code to look at is in the Scheduler class 
(in scheduler.py) and the SlotTable class (in slottable.py). When an AR 
lease is scheduled, first the schedule_ar_lease method is called which, 
in turn, calls the fit_exact method (both in the Scheduler class). 
Similarly, when a best-effort lease is scheduled, schedule_be_lease is 
called, which then calls fit_asap. Both of them rely on the "slot 
table", which is where resource reservations are actually stored and, 
more specifically, on a structure called the "availability window" which 
allows the scheduler to see what resources are available at the time 
when a lease could be scheduled.

Right now, the way the scheduler decides to allocate resources is 
basically a greedy algorithm. Once it has determined that a lease could 
potentially be scheduled at a given time, it has to choose what nodes to 
map the lease to (and this choice will affect whether other leases are 
preempted, if there will be fragmentation, etc.). The node-choosing is 
encoded in __choose_nodes in the Scheduler class. One of the things the 
policy engine will allow developers to do is to easily add their own 
resource selection algorithms (leaving the more lower-level stuff to the 
rest of Haizea).


Anyway, I hope all this helps :-) If you have any more questions or need 
additional details, please don't hesitate to ask. I realize that 
modifying some of the parts of the Haizea code (particularly the slot 
table code) might be difficult, but we're hoping this will become easier 
for developers over time.

Cheers!
-- 
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
Borja Sotomayor, University of Chicago
  Ph.D. Candidate, Department of Computer Science
  Ryerson 257-C, 1100 East 58th Street, Chicago, IL
  http://people.cs.uchicago.edu/~borja/
Haizea: http://haizea.cs.uchicago.edu/
·····························································
          "Dis maschine vill run und run!"
                -- Kurt Gödel (on the Turing Machine)
:::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::