[Haizea] Scheduling is slow
Mehdi Sheikhalishahi
mehdi.alishahi at gmail.com
Thu Oct 28 04:46:07 CDT 2010
Hello,
I am running a workload with high utilization, the scheduling is so slow,
Sanity checks are enabled to verify correctness of scheduling process,
and there is no error, but the scheduling is not progressing. For
example for this run, it is around 15 hours running.
Also I enabled VDEBUG and DEBUG, and 've seen nothing is wrong in
scheduling process.
Some information the workload trace is as the following,:
Num proc dist (num_processors:number of leases)
-----------------------------------------------
[(1, 2835), (2, 1085), (3, 8), (4, 137), (5, 11), (6, 42), (7, 87),
(8, 220), (9, 15), (10, 5), (12, 15), (14, 1), (15, 1), (16, 100),
(18, 3), (20, 4), (24, 2), (32, 3), (64, 1)]
Mem dist (memreq * num_processors):number of leases
---------------------------------------------------
[(1024, 2835), (2048, 1085), (3072, 8), (4096, 137), (5120, 11),
(6144, 42), (7168, 87), (8192, 220), (9216, 15), (10240, 5), (12288,
15), (14336, 1), (15360, 1), (16384, 100), (18432, 3), (20480, 4),
(24576, 2), (32768, 3), (65536, 1)]
SLOWDOWNS
---------
min: 1.00
10p: 1.02
25p: 7.64
med: 349.57
75p: 631.37
90p: 766.12
max: 60844.30
USERS
-----
Number of users: 28
UTILIZATION
-----------
Utilization: 91.59%
Utilization (no ramp-up/ramp-down): 88.95%
And this is Haizea status:
[2006-11-25 13:00:00.00] CLOCK Starting simulated clock
[2006-11-26 01:33:42.00] RM --- Haizea status summary ---
[2006-11-26 01:33:42.00] RM Active: 4 (4 Best-effort)
[2006-11-26 01:33:42.00] RM Done: 2 (2 Best-effort)
[2006-11-26 01:33:42.00] RM ------
[2006-11-26 01:33:42.00] RM TOTAL: 6
[2006-11-26 01:33:42.00] RM ---- End summary ----
[2006-11-26 12:22:54.00] RM --- Haizea status summary ---
[2006-11-26 12:22:54.00] RM Active: 10 (10 Best-effort)
[2006-11-26 12:22:54.00] RM Done: 8 (8 Best-effort)
[2006-11-26 12:22:54.00] RM ------
[2006-11-26 12:22:54.00] RM TOTAL: 18
[2006-11-26 12:22:54.00] RM ---- End summary ----
[2006-11-26 22:33:34.00] RM --- Haizea status summary ---
[2006-11-26 22:33:34.00] RM Active: 76 (76 Best-effort)
[2006-11-26 22:33:34.00] RM Done: 23 (23 Best-effort)
[2006-11-26 22:33:34.00] RM ------
[2006-11-26 22:33:34.00] RM TOTAL: 99
[2006-11-26 22:33:34.00] RM ---- End summary ----
[2006-11-27 08:40:07.00] RM --- Haizea status summary ---
[2006-11-27 08:40:07.00] RM Active: 7 (7 Best-effort)
[2006-11-27 08:40:07.00] RM Done: 92 (92 Best-effort)
[2006-11-27 08:40:07.00] RM ------
[2006-11-27 08:40:07.00] RM TOTAL: 99
[2006-11-27 08:40:07.00] RM ---- End summary ----
[2006-11-27 19:16:31.00] RM --- Haizea status summary ---
[2006-11-27 19:16:31.00] RM Queued: 97 (97 Best-effort)
[2006-11-27 19:16:31.00] RM Ready: 1 (1 Best-effort)
[2006-11-27 19:16:31.00] RM Active: 15 (15 Best-effort)
[2006-11-27 19:16:31.00] RM Done: 105 (105 Best-effort)
[2006-11-27 19:16:31.00] RM ------
[2006-11-27 19:16:31.00] RM TOTAL: 218
[2006-11-27 19:16:31.00] RM ---- End summary ----
[2006-11-28 10:36:09.00] RM --- Haizea status summary ---
[2006-11-28 10:36:09.00] RM Queued: 97 (97 Best-effort)
[2006-11-28 10:36:09.00] RM Ready: 1 (1 Best-effort)
[2006-11-28 10:36:09.00] RM Active: 15 (15 Best-effort)
[2006-11-28 10:36:09.00] RM Done: 107 (107 Best-effort)
[2006-11-28 10:36:09.00] RM ------
[2006-11-28 10:36:09.00] RM TOTAL: 220
[2006-11-28 10:36:09.00] RM ---- End summary ----
[2006-11-29 10:15:51.00] RM --- Haizea status summary ---
[2006-11-29 10:15:51.00] RM Queued: 97 (97 Best-effort)
[2006-11-29 10:15:51.00] RM Ready: 1 (1 Best-effort)
[2006-11-29 10:15:51.00] RM Active: 15 (15 Best-effort)
[2006-11-29 10:15:51.00] RM Done: 108 (108 Best-effort)
[2006-11-29 10:15:51.00] RM ------
[2006-11-29 10:15:51.00] RM TOTAL: 221
[2006-11-29 10:15:51.00] RM ---- End summary ----
[2006-11-29 23:25:01.00] RM --- Haizea status summary ---
[2006-11-29 23:25:01.00] RM Queued: 107 (107 Best-effort)
[2006-11-29 23:25:01.00] RM Ready: 1 (1 Best-effort)
[2006-11-29 23:25:01.00] RM Active: 15 (15 Best-effort)
[2006-11-29 23:25:01.00] RM Done: 109 (109 Best-effort)
[2006-11-29 23:25:01.00] RM ------
[2006-11-29 23:25:01.00] RM TOTAL: 232
[2006-11-29 23:25:01.00] RM ---- End summary ----
[2006-11-30 09:28:33.00] RM --- Haizea status summary ---
[2006-11-30 09:28:33.00] RM Queued: 104 (104 Best-effort)
[2006-11-30 09:28:33.00] RM Ready: 1 (1 Best-effort)
[2006-11-30 09:28:33.00] RM Active: 15 (15 Best-effort)
[2006-11-30 09:28:33.00] RM Done: 121 (121 Best-effort)
[2006-11-30 09:28:33.00] RM ------
[2006-11-30 09:28:33.00] RM TOTAL: 241
[2006-11-30 09:28:33.00] RM ---- End summary ----
[2006-11-30 20:45:45.00] RM --- Haizea status summary ---
[2006-11-30 20:45:45.00] RM Queued: 124 (124 Best-effort)
[2006-11-30 20:45:45.00] RM Ready: 1 (1 Best-effort)
[2006-11-30 20:45:45.00] RM Active: 15 (15 Best-effort)
[2006-11-30 20:45:45.00] RM Done: 124 (124 Best-effort)
[2006-11-30 20:45:45.00] RM ------
[2006-11-30 20:45:45.00] RM TOTAL: 264
[2006-11-30 20:45:45.00] RM ---- End summary ----
[2006-12-01 07:45:14.00] RM --- Haizea status summary ---
[2006-12-01 07:45:14.00] RM Queued: 131 (131 Best-effort)
[2006-12-01 07:45:14.00] RM Ready: 1 (1 Best-effort)
[2006-12-01 07:45:14.00] RM Active: 15 (15 Best-effort)
[2006-12-01 07:45:14.00] RM Done: 125 (125 Best-effort)
[2006-12-01 07:45:14.00] RM ------
[2006-12-01 07:45:14.00] RM TOTAL: 272
[2006-12-01 07:45:14.00] RM ---- End summary ----
[2006-12-01 21:09:34.00] RM --- Haizea status summary ---
[2006-12-01 21:09:34.00] RM Queued: 160 (160 Best-effort)
[2006-12-01 21:09:34.00] RM Ready: 1 (1 Best-effort)
[2006-12-01 21:09:34.00] RM Active: 15 (15 Best-effort)
[2006-12-01 21:09:34.00] RM Done: 125 (125 Best-effort)
[2006-12-01 21:09:34.00] RM ------
[2006-12-01 21:09:34.00] RM TOTAL: 301
[2006-12-01 21:09:34.00] RM ---- End summary ----
[2006-12-02 07:12:05.00] RM --- Haizea status summary ---
[2006-12-02 07:12:05.00] RM Queued: 160 (160 Best-effort)
[2006-12-02 07:12:05.00] RM Ready: 1 (1 Best-effort)
[2006-12-02 07:12:05.00] RM Active: 15 (15 Best-effort)
[2006-12-02 07:12:05.00] RM Done: 126 (126 Best-effort)
[2006-12-02 07:12:05.00] RM ------
[2006-12-02 07:12:05.00] RM TOTAL: 302
[2006-12-02 07:12:05.00] RM ---- End summary ----
[2006-12-02 17:33:49.00] RM --- Haizea status summary ---
[2006-12-02 17:33:49.00] RM Queued: 163 (163 Best-effort)
[2006-12-02 17:33:49.00] RM Ready: 1 (1 Best-effort)
[2006-12-02 17:33:49.00] RM Active: 15 (15 Best-effort)
[2006-12-02 17:33:49.00] RM Done: 137 (137 Best-effort)
[2006-12-02 17:33:49.00] RM ------
[2006-12-02 17:33:49.00] RM TOTAL: 316
[2006-12-02 17:33:49.00] RM ---- End summary ----
[2006-12-03 04:51:02.00] RM --- Haizea status summary ---
[2006-12-03 04:51:02.00] RM Queued: 2610 (2610 Best-effort)
[2006-12-03 04:51:02.00] RM Ready: 1 (1 Best-effort)
[2006-12-03 04:51:02.00] RM Active: 15 (15 Best-effort)
[2006-12-03 04:51:02.00] RM Done: 139 (139 Best-effort)
[2006-12-03 04:51:02.00] RM ------
[2006-12-03 04:51:02.00] RM TOTAL: 2765
[2006-12-03 04:51:02.00] RM ---- End summary ----
[2006-12-03 15:03:28.00] RM --- Haizea status summary ---
[2006-12-03 15:03:28.00] RM Queued: 2634 (2634 Best-effort)
[2006-12-03 15:03:28.00] RM Ready: 1 (1 Best-effort)
[2006-12-03 15:03:28.00] RM Active: 15 (15 Best-effort)
[2006-12-03 15:03:28.00] RM Done: 140 (140 Best-effort)
[2006-12-03 15:03:28.00] RM ------
[2006-12-03 15:03:28.00] RM TOTAL: 2790
[2006-12-03 15:03:28.00] RM ---- End summary ----
[2006-12-04 06:01:54.00] RM --- Haizea status summary ---
[2006-12-04 06:01:54.00] RM Queued: 2716 (2716 Best-effort)
[2006-12-04 06:01:54.00] RM Ready: 1 (1 Best-effort)
[2006-12-04 06:01:54.00] RM Active: 15 (15 Best-effort)
[2006-12-04 06:01:54.00] RM Done: 140 (140 Best-effort)
[2006-12-04 06:01:54.00] RM ------
[2006-12-04 06:01:54.00] RM TOTAL: 2872
[2006-12-04 06:01:54.00] RM ---- End summary ----
[2006-12-04 16:26:52.00] RM --- Haizea status summary ---
[2006-12-04 16:26:52.00] RM Queued: 2920 (2920 Best-effort)
[2006-12-04 16:26:52.00] RM Ready: 1 (1 Best-effort)
[2006-12-04 16:26:52.00] RM Active: 15 (15 Best-effort)
[2006-12-04 16:26:52.00] RM Done: 144 (144 Best-effort)
[2006-12-04 16:26:52.00] RM ------
[2006-12-04 16:26:52.00] RM TOTAL: 3080
[2006-12-04 16:26:52.00] RM ---- End summary ----
[2006-12-05 09:07:03.00] RM --- Haizea status summary ---
[2006-12-05 09:07:03.00] RM Queued: 2920 (2920 Best-effort)
[2006-12-05 09:07:03.00] RM Ready: 1 (1 Best-effort)
[2006-12-05 09:07:03.00] RM Active: 15 (15 Best-effort)
[2006-12-05 09:07:03.00] RM Done: 153 (153 Best-effort)
[2006-12-05 09:07:03.00] RM ------
[2006-12-05 09:07:03.00] RM TOTAL: 3089
[2006-12-05 09:07:03.00] RM ---- End summary ----
[2006-12-06 09:27:51.00] RM --- Haizea status summary ---
[2006-12-06 09:27:51.00] RM Queued: 2919 (2919 Best-effort)
[2006-12-06 09:27:51.00] RM Ready: 1 (1 Best-effort)
[2006-12-06 09:27:51.00] RM Active: 15 (15 Best-effort)
[2006-12-06 09:27:51.00] RM Done: 155 (155 Best-effort)
[2006-12-06 09:27:51.00] RM ------
[2006-12-06 09:27:51.00] RM TOTAL: 3090
[2006-12-06 09:27:51.00] RM ---- End summary ----
[2006-12-06 20:22:06.00] RM --- Haizea status summary ---
[2006-12-06 20:22:06.00] RM Queued: 2933 (2933 Best-effort)
[2006-12-06 20:22:06.00] RM Ready: 1 (1 Best-effort)
[2006-12-06 20:22:06.00] RM Active: 15 (15 Best-effort)
[2006-12-06 20:22:06.00] RM Done: 155 (155 Best-effort)
[2006-12-06 20:22:06.00] RM ------
[2006-12-06 20:22:06.00] RM TOTAL: 3104
[2006-12-06 20:22:06.00] RM ---- End summary ----
[2006-12-07 06:36:42.00] RM --- Haizea status summary ---
[2006-12-07 06:36:42.00] RM Queued: 2833 (2833 Best-effort)
[2006-12-07 06:36:42.00] RM Ready: 1 (1 Best-effort)
[2006-12-07 06:36:42.00] RM Active: 25 (25 Best-effort)
[2006-12-07 06:36:42.00] RM Done: 245 (245 Best-effort)
[2006-12-07 06:36:42.00] RM ------
[2006-12-07 06:36:42.00] RM TOTAL: 3104
[2006-12-07 06:36:42.00] RM ---- End summary ----
[2006-12-07 16:36:47.00] RM --- Haizea status summary ---
[2006-12-07 16:36:47.00] RM Queued: 2766 (2766 Best-effort)
[2006-12-07 16:36:47.00] RM Ready: 1 (1 Best-effort)
[2006-12-07 16:36:47.00] RM Active: 36 (36 Best-effort)
[2006-12-07 16:36:47.00] RM Done: 316 (316 Best-effort)
[2006-12-07 16:36:47.00] RM ------
[2006-12-07 16:36:47.00] RM TOTAL: 3119
[2006-12-07 16:36:47.00] RM ---- End summary ----
[2006-12-08 03:55:16.00] RM --- Haizea status summary ---
[2006-12-08 03:55:16.00] RM Queued: 2568 (2568 Best-effort)
[2006-12-08 03:55:16.00] RM Ready: 1 (1 Best-effort)
[2006-12-08 03:55:16.00] RM Active: 35 (35 Best-effort)
[2006-12-08 03:55:16.00] RM Done: 536 (536 Best-effort)
[2006-12-08 03:55:16.00] RM ------
[2006-12-08 03:55:16.00] RM TOTAL: 3140
[2006-12-08 03:55:16.00] RM ---- End summary ----
[2006-12-08 14:11:38.00] RM --- Haizea status summary ---
[2006-12-08 14:11:38.00] RM Queued: 2540 (2540 Best-effort)
[2006-12-08 14:11:38.00] RM Ready: 1 (1 Best-effort)
[2006-12-08 14:11:38.00] RM Active: 31 (31 Best-effort)
[2006-12-08 14:11:38.00] RM Done: 576 (576 Best-effort)
[2006-12-08 14:11:38.00] RM ------
[2006-12-08 14:11:38.00] RM TOTAL: 3148
[2006-12-08 14:11:38.00] RM ---- End summary ----
[2006-12-09 00:11:40.00] RM --- Haizea status summary ---
[2006-12-09 00:11:40.00] RM Queued: 2340 (2340 Best-effort)
[2006-12-09 00:11:40.00] RM Ready: 1 (1 Best-effort)
[2006-12-09 00:11:40.00] RM Active: 28 (28 Best-effort)
[2006-12-09 00:11:40.00] RM Done: 788 (788 Best-effort)
[2006-12-09 00:11:40.00] RM ------
[2006-12-09 00:11:40.00] RM TOTAL: 3157
[2006-12-09 00:11:40.00] RM ---- End summary ----
Regards,
Mehdi
More information about the Haizea
mailing list