Truss on process with high CPU usage shows 90 % time() calls

  • Permalink
  • submit to reddit
  • Email
  • Follow


Hi,

We've got a performance problem on an application (webmethods
Enterprise Server -> and the process awbroker) on Solaris 8. The same
configuration on a different machine works just fine and we can't find
any configuration differences in the application itself... What is
different the network configuration, but I'll return to that in a
minute.

The only reason for posting to this group is that when we did a truss
-p on the process there was a incredible amount of calls to time().
(Unfortunately we did not do a trusss -c to have exact numbers on
this: we may do some controlled experiments next week where we'll
measure this exactly.) A graph of vmstat activity averaged over 100
seconds shows that at the moment of the truss there were around 80k
system calls/second.

So what does the application do? One one server it reads a big file in
XML format that is stored in a queue (on the broker). An adapter on a
distant server retrieves the documents in that queue. The problem is
that it retrieves the documents very slowly compared to another
server. The physical difference is that the broker server and the
adapter that retrieves the documents are on the same LAN  with a
simple switch next to each other (in the FAST case); in the SLOW case
the two are in a VLAN on two separate sites situated approximately 1
km apart.

Of course, it's hard to know what the broker is doing in this case...
But, if anyone has a clue to why a process communicating with the
distant server would call time() something like 70k times/second we'd
appreciate it!

Thanks in advance for any help or ideas.

Best regards,

Frank Olsen
0
Reply folsen 12/17/2003 3:15:46 PM

See related articles to this posting

Some additional info here would really help, but I'll
proffer some guesses:

* is the app using UDP and using a clumsy retransmit scheme?
* is the app failing to handle possible packet fragmentation?
* does the app check timestamps on things; are the machines 
  in sync wrt time?

- Bart

folsen@acm.org (Frank Olsen) wrote in message news:<53cb967a.0312170715.56c6bc70@posting.google.com>...
> Hi,
> 
> We've got a performance problem on an application (webmethods
> Enterprise Server -> and the process awbroker) on Solaris 8. The same
> configuration on a different machine works just fine and we can't find
> any configuration differences in the application itself... What is
> different the network configuration, but I'll return to that in a
> minute.
> 
> The only reason for posting to this group is that when we did a truss
> -p on the process there was a incredible amount of calls to time().
> (Unfortunately we did not do a trusss -c to have exact numbers on
> this: we may do some controlled experiments next week where we'll
> measure this exactly.) A graph of vmstat activity averaged over 100
> seconds shows that at the moment of the truss there were around 80k
> system calls/second.
> 
> So what does the application do? One one server it reads a big file in
> XML format that is stored in a queue (on the broker). An adapter on a
> distant server retrieves the documents in that queue. The problem is
> that it retrieves the documents very slowly compared to another
> server. The physical difference is that the broker server and the
> adapter that retrieves the documents are on the same LAN  with a
> simple switch next to each other (in the FAST case); in the SLOW case
> the two are in a VLAN on two separate sites situated approximately 1
> km apart.
> 
> Of course, it's hard to know what the broker is doing in this case...
> But, if anyone has a clue to why a process communicating with the
> distant server would call time() something like 70k times/second we'd
> appreciate it!
> 
> Thanks in advance for any help or ideas.
> 
> Best regards,
> 
> Frank Olsen
0
Reply barts 12/19/2003 2:00:18 AM
comp.unix.solaris 25703 articles. 86 followers. Post

1 Replies
275 Views

Similar Articles

[PageSpeed] 51

  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Keeping VoIP calls up while high IP Input process cpu time
Hello, I was having a problem with a router that I'm not sure how's the best way to prevent it from happening again. I have a MC3810 running 12.3, and it provides a PRI and NAT through a T1 data connection. One of the computers running through the NAT got a virus and started opening up thousands of connections to port 443 per second. This caused the IP Input process to max out the CPU time, and as a side effect VoIP calls couldn't go through. I fixed the problem with an access list so that port won't be a problem in the future, however, I'm sure at some time so...

cpu time usage of multi threaded process and single threaded process.
these are 2 programs for matrix multiplication. :- 1st one is implemented using multiple threading and 2nd using single threading. Process' Cpu time for both comes out to be same ..... but should be less for multi threaded process. Plzz help why Cpu time is same.... ? program1 using multiple threading :- #include<signal.h> #include<stdio.h> #include<sys/types.h> #include<unistd.h> #include<pthread.h> #include<time.h> // decleared globally so can be used by all threads through resorce shareing and dont get destroyed on exit of threads...

system idle process high cpu usage
hello I was having problems with 100% cpu usage & at the Microsoft site they specified to install the patch to overcome the problem & i did. Now System idle Process shows 100% cpu usage. Please give the solution at the earliest. Thanking you. -- matrex722 ...

process memory usage, CPU Time by given
ello, How ACE allows to access a process memory usage, CPU Time by given PID? Thanks. ...

High (80-90%) CPU usage on W98
I have a 1989 Dell Dimension XPS T500MHz w/ 512MB RAM, running Win98. Recently, the system has been extremely processor bound, as demonstraded by System Monitor (Kernel Proc Usage tends to "redline"). I have been unable to identify the culprit, but a CTL-ALT-DEL does shows tasks by the name of "ccd", "cfd" and "mdm". When I kill these tasks (usually the are "not responding"), the perfromance does improve. I have searched high and low in documentation and on the net, trying to identify what these tasks are. They do NOT appear in the mscon...

How do I get percentage CPU usage of processes for a specified time interval?
How do I get percentage CPU usage of processes for a specified time interval? I'm wish to write a script, part of which displays the top 10 processes by percentage processor usage over a 30 second interval. I know "topas" can show this, but I need to be able to process the information in a script. The output from "ps" doesn't seem to give the neccessary resolution... Can anybody give me some pointers as to how this can be done? Many Thanks, =Adrian= Adrian Davis wrote: > How do I get percentage CPU usage of processes for a specified time > interval? &g...

How can a process consume kernel CPU time without making system calls?
I'm trying to track down what is going on with a particular rogue application process on a customer site running AIX 4.3.3. Under defined circumstances, the application process can be fired up and while seemingly doing nothing will soar to the top of the topas list of jobs whilst taking the system from largely idle to heavily CPU bound. When this happens, it is always the kernel histogram that goes highest, and if I time(1) the process from start to when I break it out of the resource hogging section I get output like this: usr:5.41 sys:19.53 total:24.94 elapsed:0m31.11s It seemed...

Comcast cable internet -> CPU mostly 100% (tool to show per-process usage)?
Ever since I got my High Speed Digital Internet access from Comcast/Verizon, my CPU runs mostly at 100% and I frequently have long (5-30 second) lockups for no reason (sometimes coincides with changing window focus). I am running Windows 98 and the crummy taks process program (Ctl-Alt-Del) does not show CPU usage per-process so I can't tell what is hogging it all. Is there a good freeware/trialware performance monitor that will show this for W98? Anybody get Virzon/Comcast and then figure this out? Is there a better place to post? > Ever since I got my High Speed Digital ...

High Execution Time--Low CPU time for statements
Scenario: new functionality was recently deployed in a 3-tier business application. Since deployment, CPU use has shot up to very high levels. Using event monitors to try to track down the culprits hasn't fingered any bad query in particular who's hogging CPU. One interesting anomaly is that there is a single type of query that averages 70 seconds of real time (with highs of 100-300 seconds) while expending zero seconds of CPU. The query in question takes the form "SELECT * FROM TABLE WHERE COLUMN = 'VALUE'" where the value is the first element in the prima...