Quantcast

4.1 CPU usage

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

4.1 CPU usage

Jeff Siddall
I have noticed that since upgrading to mh 4.1 CPU usage is significantly
higher (~84% currently) than it was the last time I checked when running
mh 3.

So I ran a benchmark and found this:

Benchmark report.  Loop count=9516.  The following is in milliseconds
    OTHER                 avg= 5.26 total=50034
    USER                  avg=126.35 total=1202326

I don't know where 126.35 comes from because my two worst files were
avg= 7.00 and avg= 3.17 respectively.  All the rest were below 1.

One other thing I noticed is that some object type stuff (ex: tie_items)
was in the loopcode subs, which I thought would have been pulled out.  I
also noticed some subs defined in my files were still in the loopcode too.

That leaves me a couple of questions:

1. Should tie_items and subs have been automatically pulled out of the
loopcode?

2. Why does the USER benchmark show an average of 126.35 ms when the
worst file was about 7, and the grand total was below 15?

Thanks,

Jeff

------------------------------------------------------------------------------
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

H Plato
While i don't have any suggestions about #2, I wonder if the perltidy issue is impacting things. Can you edit the files and ensure that object methods don't break over multiple lines and see if that changes anything?

Sent from my mobile device.

> On Mar 4, 2016, at 2:19 PM, Jeff Siddall <[hidden email]> wrote:
>
> I have noticed that since upgrading to mh 4.1 CPU usage is significantly
> higher (~84% currently) than it was the last time I checked when running
> mh 3.
>
> So I ran a benchmark and found this:
>
> Benchmark report.  Loop count=9516.  The following is in milliseconds
>    OTHER                 avg= 5.26 total=50034
>    USER                  avg=126.35 total=1202326
>
> I don't know where 126.35 comes from because my two worst files were
> avg= 7.00 and avg= 3.17 respectively.  All the rest were below 1.
>
> One other thing I noticed is that some object type stuff (ex: tie_items)
> was in the loopcode subs, which I thought would have been pulled out.  I
> also noticed some subs defined in my files were still in the loopcode too.
>
> That leaves me a couple of questions:
>
> 1. Should tie_items and subs have been automatically pulled out of the
> loopcode?
>
> 2. Why does the USER benchmark show an average of 126.35 ms when the
> worst file was about 7, and the grand total was below 15?
>
> Thanks,
>
> Jeff
>
> ------------------------------------------------------------------------------
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>

------------------------------------------------------------------------------
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

Tobias Sachs
In reply to this post by Jeff Siddall
Hello Jeff,

high CPU usage may be caused by the constant polling from ia7.
For me mh CPU usage goes way up if I have the ia7 web-interface open.
Without any browser open mh is currently using ~4%
My ia7 start page makes mh run at ~10%
ia7 "Browse Groups" uses ~15%
ia7 "floorplan" for me uses around ~20%
Opening multiple tabs makes mh CPU usage much worse.

Tobi

Am 04.03.2016 um 22:19 schrieb Jeff Siddall:

> I have noticed that since upgrading to mh 4.1 CPU usage is significantly
> higher (~84% currently) than it was the last time I checked when running
> mh 3.
>
> So I ran a benchmark and found this:
>
> Benchmark report.  Loop count=9516.  The following is in milliseconds
>      OTHER                 avg= 5.26 total=50034
>      USER                  avg=126.35 total=1202326
>
> I don't know where 126.35 comes from because my two worst files were
> avg= 7.00 and avg= 3.17 respectively.  All the rest were below 1.
>
> One other thing I noticed is that some object type stuff (ex: tie_items)
> was in the loopcode subs, which I thought would have been pulled out.  I
> also noticed some subs defined in my files were still in the loopcode too.
>
> That leaves me a couple of questions:
>
> 1. Should tie_items and subs have been automatically pulled out of the
> loopcode?
>
> 2. Why does the USER benchmark show an average of 126.35 ms when the
> worst file was about 7, and the grand total was below 15?
>
> Thanks,
>
> Jeff
>
> ------------------------------------------------------------------------------
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>


------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

H Plato
Just a clarification, IA7 doesn’t constantly poll itself. There is a new method added called long_poll which only returns changed data. This is great since it minimizes the resources needed in the client browser, but json_server.pl does have to check each loop if any items have changed.  I don’t see quite the same level of CPU utilization on my server (quad core pentium), but since MH runs on nearly everything it might be good to see if this creates an impact for anyone.

> On Mar 9, 2016, at 11:13 AM, Tobias Sachs <[hidden email]> wrote:
>
> Hello Jeff,
>
> high CPU usage may be caused by the constant polling from ia7.
> For me mh CPU usage goes way up if I have the ia7 web-interface open.
> Without any browser open mh is currently using ~4%
> My ia7 start page makes mh run at ~10%
> ia7 "Browse Groups" uses ~15%
> ia7 "floorplan" for me uses around ~20%
> Opening multiple tabs makes mh CPU usage much worse.
>
> Tobi
>
> Am 04.03.2016 um 22:19 schrieb Jeff Siddall:
>> I have noticed that since upgrading to mh 4.1 CPU usage is significantly
>> higher (~84% currently) than it was the last time I checked when running
>> mh 3.
>>
>> So I ran a benchmark and found this:
>>
>> Benchmark report.  Loop count=9516.  The following is in milliseconds
>>     OTHER                 avg= 5.26 total=50034
>>     USER                  avg=126.35 total=1202326
>>
>> I don't know where 126.35 comes from because my two worst files were
>> avg= 7.00 and avg= 3.17 respectively.  All the rest were below 1.
>>
>> One other thing I noticed is that some object type stuff (ex: tie_items)
>> was in the loopcode subs, which I thought would have been pulled out.  I
>> also noticed some subs defined in my files were still in the loopcode too.
>>
>> That leaves me a couple of questions:
>>
>> 1. Should tie_items and subs have been automatically pulled out of the
>> loopcode?
>>
>> 2. Why does the USER benchmark show an average of 126.35 ms when the
>> worst file was about 7, and the grand total was below 15?
>>
>> Thanks,
>>
>> Jeff
>>
>> ------------------------------------------------------------------------------
>> ________________________________________________________
>> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>>
>
>
> ------------------------------------------------------------------------------
> Transform Data into Opportunity.
> Accelerate data analysis in your applications with
> Intel Data Analytics Acceleration Library.
> Click to learn more.
> http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
> ________________________________________________________
> To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users
>


------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

Giles Godart-Brown
I'm running MH 4.1 on a Raspberry Pi2B which has a much more modest processor.

Here is the output of top before I start ia7. I use a proxy for X10 comms, hence 2 processes;

PID         USER      PR  NI      VIRT          RES       SHR S      %CPU %MEM         TIME+ COMMAND   
13965     pi           20   0       61492      51572   5936 S      16.2  5.4               0:13.23 mh        
13969     pi           20   0       41080      32560   5044 S        2.3  3.4               0:04.71 perl      

And now I start ia7.

 PID      USER      PR  NI    VIRT    RES         SHR S    %CPU %MEM     TIME+ COMMAND   
13965 pi            20   0      62192  52184   5944 R  49.3    5.5            1:00.44 mh        
13969 pi            20   0      41080  32560   5044 S   1.7     3.4             0:09.55 perl      

When I close  the ia7  page it returns to about 17% again

Giles

On 11/03/2016 22:50, H Plato wrote:
Just a clarification, IA7 doesn’t constantly poll itself. There is a new method added called long_poll which only returns changed data. This is great since it minimizes the resources needed in the client browser, but json_server.pl does have to check each loop if any items have changed.  I don’t see quite the same level of CPU utilization on my server (quad core pentium), but since MH runs on nearly everything it might be good to see if this creates an impact for anyone.

On Mar 9, 2016, at 11:13 AM, Tobias Sachs [hidden email] wrote:

Hello Jeff,

high CPU usage may be caused by the constant polling from ia7.
For me mh CPU usage goes way up if I have the ia7 web-interface open.
Without any browser open mh is currently using ~4%
My ia7 start page makes mh run at ~10%
ia7 "Browse Groups" uses ~15%
ia7 "floorplan" for me uses around ~20%
Opening multiple tabs makes mh CPU usage much worse.

Tobi

Am 04.03.2016 um 22:19 schrieb Jeff Siddall:
I have noticed that since upgrading to mh 4.1 CPU usage is significantly
higher (~84% currently) than it was the last time I checked when running
mh 3.

So I ran a benchmark and found this:

Benchmark report.  Loop count=9516.  The following is in milliseconds
    OTHER                 avg= 5.26 total=50034
    USER                  avg=126.35 total=1202326

I don't know where 126.35 comes from because my two worst files were
avg= 7.00 and avg= 3.17 respectively.  All the rest were below 1.

One other thing I noticed is that some object type stuff (ex: tie_items)
was in the loopcode subs, which I thought would have been pulled out.  I
also noticed some subs defined in my files were still in the loopcode too.

That leaves me a couple of questions:

1. Should tie_items and subs have been automatically pulled out of the
loopcode?

2. Why does the USER benchmark show an average of 126.35 ms when the
worst file was about 7, and the grand total was below 15?

Thanks,

Jeff

------------------------------------------------------------------------------
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users


------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users


------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users



------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785111&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

Jeff Siddall
On 03/12/2016 06:09 AM, Giles Godart-Brown wrote:
> I'm running MH 4.1 on a Raspberry Pi2B which has a much more modest
> processor.

Yeah, looks like this is at least part of it.  When I have no IA7 pages
open I am at ~33% CPU (still significant on a D510 CPU) but it jumps up
to 86% when I open one IA7 page.  More pages doesn't seem to affect
anything as the CPU is already saturated.

Still, approx. 3X the CPU usage with IA7 is a big jump.  Can
json_server.pl be tuned at all?

Jeff

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 4.1 CPU usage

H Plato
So I’ve been digging into this old issue.  I thought it might be gzip, since the json data is gzipped before sending to the client to reduce the data transfer. Enabling or disabling this function doesn’t seem have a discernible impact.

Does anyone have any experience with profiling perl code? I’ve been using Devel::NYTProf, which shows what is being called, but I’m not that clear on how I can use this to zero in on what’s driving the CPU usage.

Running for a few minutes I got this for output:
116842154.3s54.3s main::CORE:sselect (opcode)
1278111.22s3.51s main::build_parent_table
4217411997ms1.20s Group::list
13501863672ms816ms main::get_object_by_name
54622628ms1.19s main::list_code_webnames
261511483ms6.22s main::json_get
5751011441ms553ms Generic_Item::get_object_name
29169144438ms511ms main::get_tickcount
16046010145334ms334msTie::ExtraHash::FETCH
1022521314ms644ms main::json_object_detail
96011288ms57.6s main::<a href="file:///Users/howard/Develop/mh/system/bin/nytprof/(eval%20140)[mh-3490]-272-line.html#7013" style="color: rgb(109, 0, 230);" class="">loop_code
3194571568275ms275ms main::CORE:match (opcode)
96011267ms544ms main::<a href="file:///Users/howard/Develop/mh/system/bin/nytprof/(eval%20140)[mh-3490]-272-line.html#6545" style="color: rgb(109, 0, 230);" class="">items_loopcode
111259ms259ms main::CORE:ghbyname (opcode)
26153434249ms6.53s main::json

and for json_server:
CallsPFExclusive
Time
Inclusive
Time
Subroutine
1278111.22s3.51smain::build_parent_table
261511483ms6.22smain::json_get
1022521314ms644msmain::json_object_detail
26153434249ms6.53smain::json
1310931102ms102msmain::filter_object
26151126.6ms26.6msmain::json_get_sub_element
28803114.9ms14.9msmain::json_table_put_header
28803113.9ms13.9msmain::json_table_put_data
9601111.6ms25.0msmain::json_table_push
960116.19ms6.19msmain::json_table_create
1112.62ms43.3msmain::BEGIN@46.30
1112.61ms26.6msmain::BEGIN@43
5711736µs112msmain::json_page


Now I did find an unnecessary sort, which seemed to have improved things (the previous run, json_get was double for time), but I wasn’t clever enough to have an actual scripted set of actions, go comparable data isn’t that valid.


On Mar 21, 2016, at 8:33 AM, Jeff Siddall <[hidden email]> wrote:

On 03/12/2016 06:09 AM, Giles Godart-Brown wrote:
I'm running MH 4.1 on a Raspberry Pi2B which has a much more modest
processor.

Yeah, looks like this is at least part of it.  When I have no IA7 pages
open I am at ~33% CPU (still significant on a D510 CPU) but it jumps up
to 86% when I open one IA7 page.  More pages doesn't seem to affect
anything as the CPU is already saturated.

Still, approx. 3X the CPU usage with IA7 is a big jump.  Can
json_server.pl be tuned at all?

Jeff

------------------------------------------------------------------------------
Transform Data into Opportunity.
Accelerate data analysis in your applications with
Intel Data Analytics Acceleration Library.
Click to learn more.
http://pubads.g.doubleclick.net/gampad/clk?id=278785351&iu=/4140
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users



------------------------------------------------------------------------------
What NetFlow Analyzer can do for you? Monitors network bandwidth and traffic
patterns at an interface-level. Reveals which users, apps, and protocols are
consuming the most bandwidth. Provides multi-vendor support for NetFlow,
J-Flow, sFlow and other flows. Make informed decisions using capacity planning
reports.http://sdm.link/zohodev2dev
________________________________________________________
To unsubscribe from this list, go to: https://lists.sourceforge.net/lists/listinfo/misterhouse-users

Loading...