[Haizea] Developer Docs...

[J Arshad]

Hi,

I've been looking into the package to understand and identify the places 
where I want to make changes to achieve my objective. Below are some of 
the questions I have, could anyone please suggest answers to these...

Firstly, One of the files I want to change is the sample_trace.conf i.e. 
I want to have resources with different capabilities. Currently, all the
nodes are assumed to have the same characteristics e.g. CPU,Mem etc. using 
the "resources" option in the sample_trace.conf( please correct me if I am 
wrong). However, I want to add an extra characteristic e.g. availability 
of "MRI device" with the resources and for this, I assume that not all the 
resources will have such device. I tried to have multiple "resources" 
options but, of course, that does not seem to work. Could you please 
suggest something in this regard...

Secondly, I believe the "changepoints" logic for the Best Effort leases 
comes into play with "suspension" being true and/or the resource is 
migrated/resumed. Please correct me if I am wrong again...

Finally, can anyone please suggest ways to analyze the output of the 
simulation.


Thanks in advance

Cheers,

Junaid

On Wed, 10 Dec 2008, Borja Sotomayor wrote:

> Hi,
>
>> For now, can you please tell me the modules to look for, for lease
>> acceptance and resource allocation process to make any changes that I
>> might want...
>
> Sure! For lease acceptance, the starting point is the rm.py module (the
> "Resource Manager"; this is essentially the root module of Haizea. There
> are some comments in that module that you may find helpful to navigate
> the code).
>
> The RM module can have several "frontends", which is where lease
> requests are taken from. It is the frontends' responsibility to get the
> lease requests and hand them over to the RM, but they make no decision
> regarding whether to accept the request or not (this would be the RM's
> responsibility). When running in simulation, Haizea uses the
> TracefileFrontend class to read from a LWF file (see
> resourcemanager/frontends/tracefile.py and traces/readers.py). When
> running with OpenNebula, Haizea uses the OpenNebulaFrontend, which can
> access the OpenNebula database to obtain lease requests.
>
> The RM will periodically check to see if there are any new requests in
> the frontends. This is done in method process_requests of the
> ResourceManager class (in rm.py). This function has some in-code
> documentation. As you'll see, it basically just collects the pending
> requests and hands them off to the scheduler (method request_lease of
> the Scheduler class in scheduler.py), which will process pending
> requests when the scheduling function is called (method "schedule" of
> the Schedule class)
>
> As you'll see, Haizea currently has an "accept all" policy: all AR
> leases are accepted (as long as there is enough resources to satisfy the
> request) and all best-effort leases are placed in the queue in FCFS
> order (although this order is tweaked using backfilling). The upcoming
> policy engine will make it easy for developers to plug in their own
> policies (e.g., you could write a simple python module that checks if
> the AR lease is requested at least N hours in advance; if not, the lease
> request is rejected).
>
> If you want to modify lease acceptance, I'd say there are two ways of
> going about it:
>
> * If you're going to decide acceptance based on criteria unrelated to
> the current schedule (e.g., what user is requesting the lease, the
> duration of the lease, etc.), I'd say the appropriate place for this
> would be the process_requests method of the ResourceManager class.
>
> * If you're going to decide acceptance based on criteria related to the
> current schedule (e.g., don't accept an AR lease if the current number
> of ARs scheduled uses up more than N% of my resources, etc.), the most
> appropriate place would be the request_lease method of the Scheduler class.
>
>
> The code for resource allocation is still a bit convoluted, and not as
> well commented as other parts of Haizea (although I hope this will
> gradually improve). The main code to look at is in the Scheduler class
> (in scheduler.py) and the SlotTable class (in slottable.py). When an AR
> lease is scheduled, first the schedule_ar_lease method is called which,
> in turn, calls the fit_exact method (both in the Scheduler class).
> Similarly, when a best-effort lease is scheduled, schedule_be_lease is
> called, which then calls fit_asap. Both of them rely on the "slot
> table", which is where resource reservations are actually stored and,
> more specifically, on a structure called the "availability window" which
> allows the scheduler to see what resources are available at the time
> when a lease could be scheduled.
>
> Right now, the way the scheduler decides to allocate resources is
> basically a greedy algorithm. Once it has determined that a lease could
> potentially be scheduled at a given time, it has to choose what nodes to
> map the lease to (and this choice will affect whether other leases are
> preempted, if there will be fragmentation, etc.). The node-choosing is
> encoded in __choose_nodes in the Scheduler class. One of the things the
> policy engine will allow developers to do is to easily add their own
> resource selection algorithms (leaving the more lower-level stuff to the
> rest of Haizea).
>
>
> Anyway, I hope all this helps :-) If you have any more questions or need
> additional details, please don't hesitate to ask. I realize that
> modifying some of the parts of the Haizea code (particularly the slot
> table code) might be difficult, but we're hoping this will become easier
> for developers over time.
>
> Cheers!
>