Help: Core was generated - comp.unix.programmer

Hi, all, One core was generated during execution. According to the pstack result, it seems that the core came from thread 19 which was running operator() method. I was confused because most core stack I have seens was caused by kill, abort or others, The last method called in thread was operator() instead of kill, abort... This core was generated once, and I was unable to reproduce it again. Could anyone give me some hints on how to finger out the root cause? Thanks in advance. Pstack core: execution workshop_c++-11.0_p2/c++filt () core 'core' of 27413: controller.x -roc=aaa -cluster=server1 ----------------- lwp# 1 / thread# 1 -------------------- fd2c4aa0 __lwp_park (5e800, 5e7d8, 0, 0, 0, 0) + 14 fd2beae8 cond_wait_queue (5e800, 5e7d8, 0, 0, 0, 0) + 28 fd2bf068 cond_wait (5e800, 5e7d8, feaaeed4, 0, ff3f42f0, 0) + 10 fd2bf0a4 pthread_cond_wait (5e800, 5e7d8, feaaedcc, 0, ff3f42f0, 15b74) + 8 .... feaac330 main (7, ffbfe21c, ffbfe23c, 35000, fcdfe940, fcdfe980) + 40 00018ba8 _start (0, 0, 0, 0, 0, 0) + 108 ----------------- lwp# 2 / thread# 2 -------------------- fd2c4aa0 __lwp_park (fc5fb960, 805c8, 0, 0, 0, 0) + 14 fd2beae8 cond_wait_queue (fc5fb960, 805c8, 0, 0, 0, 0) + 28 fd2bf068 cond_wait (fc5fb960, 805c8, ffffffff, fffffff8, ffffffe0, fc5fb979) + 10 fd2bf0a4 pthread_cond_wait (fc5fb960, 805c8, 4356, ff000000, 0, 4000) + 8 fdcb7684 int ACE_OS::cond_timedwait (_pthread_cond*,_pthread_mutex*,ACE_Time_Value*) (fc5fb960, 805c8, 0, 0, 0, 0) + 8c fdcb7d6c int ACE_Condition_Thread_Mutex::wait(ACE_Thread_Mutex&,const ACE_Time_Value*) (fc5fb960, 805c8, 0, 0, fcd43a00, 1000) + 24 ...... fe05011c int TAO_ORB_Core::run(ACE_Time_Value*,int) (7c518, 0, 0, 12, fcd43a00, 1000) + 364 fe04037c void CORBA::ORB::run(ACE_Time_Value*) (80830, 0, 0, 10, 10b4, fd2f3b00) + 2c fe0402f8 void CORBA::ORB::run() (80830, 80830, 0, 0, fcd43a00, 10) + 10 fead2f74 void* CorbaThreadPool::poolThread(void*) (9f7e8, fc5fc000, 0, 0, fead2d80, 1) + 1f4 fd2c49fc _lwp_start (0, 0, 0, 0, 0, 0) ----------------- lwp# 3 / thread# 3 -------------------- .... ----------------- lwp# 18 / thread# 18 -------------------- #The stack trace in lwp #3---lwp #18 are the same as that of lwp#2 ----------------- lwp# 19 / thread# 19 -------------------- ff319df4 bool std::less<FMK_String>::operator()(const FMK_String&,const FMK_String&)const (ff27fce9, 4d68, fbbf9be4, 46, cd5d4, ffc9ff46) + 14 ff264e8c __rwstd::__rb_tree<FMK_String,std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,__rwstd::__select1st<std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,FMK_String>,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >::iterator __rwstd::__rb_tree<FMK_String,std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,__rwstd::__select1st<std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,FMK_String>,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >::find(const FMK_String&)const (fbbf956c, ff27fccc, fbbf9be4, 1000, 0, 0) + 74 ff261e54 __rwstd::__rb_tree<FMK_String,std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,__rwstd::__select1st<std::pair<const FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement>,FMK_String>,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >::const_iterator std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >::find(const FMK_String&)const (fbbf95d8, ff27fccc, fbbf9be4, 0, ff3f42f0, 0) + 24 ff261d44 bool RW_VMapAssoc<std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >,RWTValMap<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::less<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement> >,FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::less<FMK_String> >::contains(const FMK_String&)const (ff27fccc, fbbf9be4, fbbf9d50, fcd48a00, fd2f47e0, 0) + 2c ff262bec void TokenManager<FMKTokenSupervisor>::Unlisten(const FMK_String&) (ff27fcc8, fbbf9be4, 4, fd5798f0, 0, fc00) + 184 ff30f3d8 void Controller_i::logout(const wtk::FMK_ItString&,const wtk::FMK_ItUserContext&) (a1680, fbbfa1f4, fbbfa1e4, fcd48a00, fd2f1818, 0) + 4f8 0001fe6c void POA_wisci_wicl::Controller_tie<Controller_i>::logout (const wtk::FMK_ItString&,const wtk::FMK_ItUserContext&) (b0698, fbbfa1f4, fbbfa1e4, ff025340, fd2ee308, fd2f7870) + 3c ff2d15f4 void POA_wisci_wicl::logout_Controller::execute() (fbbfa1c4, fbbfab70, 1, 3, fbbfa3e0, 0) + 5c ................. fdd42538 int ACE_Reactor::handle_events(ACE_Time_Value*) (764d8, 0, 0, 0, 0, 0) + 40 fe05011c int TAO_ORB_Core::run(ACE_Time_Value*,int) (7c518, 0, 0, 12, fcd48a00, 1000) + 364 fe04037c void CORBA::ORB::run(ACE_Time_Value*) (80830, 0, 0, 10, 10b4, fd2f3b00) + 2c fe0402f8 void CORBA::ORB::run() (80830, 80830, 0, 0, fcd48a00, 10) + 10 fead2f74 void* CorbaThreadPool::poolThread(void*) (a0328, fbbfc000, 0, 0, fead2d80, 1) + 1f4 fd2c49fc _lwp_start (0, 0, 0, 0, 0, 0) ----------------- lwp# 20 / thread# 20 -------------------- # Same to the thread #2 ----------------- lwp# 21 / thread# 21 -------------------- #Same to the thread #2 ----------------- lwp# 22 / thread# 22 -------------------- fd2c4aa0 __lwp_park (af5c8, 7fdc8, 0, 0, 0, 0) + 14 fd2beae8 cond_wait_queue (af5c8, 7fdc8, 0, 0, 0, 0) + 28 fd2bf068 cond_wait (af5c8, 7fdc8, 0, 1000, 0, 0) + 10 fd2bf0a4 pthread_cond_wait (af5c8, 7fdc8, fb8f9dac, fb8f9ef0, 1, 0) + 8 fdcb7684 int ACE_OS::cond_timedwait (_pthread_cond*,_pthread_mutex*,ACE_Time_Value*) (af5c8, 7fdc8, 0, 0, 0, d3068) + 8c fdcb7d6c int ACE_Condition_Thread_Mutex::wait(ACE_Thread_Mutex&,const ACE_Time_Value*) (af5c8, 7fdc8, 0, 5fa058, 5fa058, 2) + 24 fdcb7db8 int ACE_Condition_Thread_Mutex::wait(const ACE_Time_Value*) (af5c8, 0, 1, 5fa234, 0, 0) + 20 fe017d14 int TAO_LF_Follower::wait(ACE_Time_Value*) (af5b8, 0, af5b8, 5fa234, 0, 5f) + 1c fe01675c int TAO_Leader_Follower::wait_for_event (TAO_LF_Event*,TAO_Transport*,ACE_Time_Value*) (7fdc0, fb8f99b8, 5f9f60, 0, 0, 2) + 36c fe0bc230 int TAO_Wait_On_Leader_Follower::wait (ACE_Time_Value*,TAO_Synch_Reply_Dispatcher&) (d3038, 0, fb8f99ac, 5fa234, 1, 0) + 68 fe07ed24 TAO::Invocation_Status TAO::Synch_Twoway_Invocation::wait_for_reply (ACE_Time_Value*,TAO_Synch_Reply_Dispatcher&,TAO_Bind_Dispatcher_Guard&) (fb8f9cc8, 0, fb8f99ac, fb8f9978, 0, 0) + 84 ................. fdd42538 int ACE_Reactor::handle_events(ACE_Time_Value*) (764d8, 0, 0, 0, ff3f4910, 821) + 40 fe05011c int TAO_ORB_Core::run(ACE_Time_Value*,int) (7c518, 0, 0, 0, ff3f42f0, 0) + 364 fe04037c void CORBA::ORB::run(ACE_Time_Value*) (80830, 0, ff000000, 0, 0, fcd4a200) + 2c fe0402f8 void CORBA::ORB::run() (80830, 80830, 0, 0, fcd4a200, 1) + 10 fead3708 void* CorbaThreadPool::adminThread(void*) (0, fb8fc000, 0, 0, fead3570, 1) + 198 fd2c49fc _lwp_start (0, 0, 0, 0, 0, 0) ----------------- lwp# 1522 / thread# 1522 -------------------- fd2c4aa0 __lwp_park (ff27fdb8, ff27fd90, fb4fb7f8, 1, 0, 0) + 14 fd2beae8 cond_wait_queue (ff27fdb8, ff27fd90, fb4fb7f8, 0, 0, 0) + 28 fd2bef60 cond_wait_common (ff27fdb8, ff27fd90, fb4fb7f8, 0, 0, 0) + 298 fd2bf0f8 _cond_timedwait (ff27fdb8, ff27fd90, fb4fb944, 0, ff3f42f0, 0) + 34 fd2bf1ec cond_timedwait (ff27fdb8, ff27fd90, fb4fb944, 0, 0, fb4fb94c) + 14 fd2bf22c pthread_cond_timedwait (ff27fdb8, ff27fd90, fb4fb944, 0, ff27fd88, 0) + c fe174164 RWWaitStatus RWCondition::wait(unsigned long) (ff27fdb0, 1388, fd82b22c, 0, fe189b18, fe1740c0) + 84 fe1771a8 RWWaitStatus RWSemaphore::acquire(unsigned long) (ff27fd80, 1388, 0, fe189b18, ffffffff, ff27fdb0) + 98 ff261438 void TokenManager<FMKTokenSupervisor>::TokenEventMonitor() (ff27fcc8, 0, 0, 0, 0, 0) + 120 ff267090 void RWTFunctor0MImp<TokenManager<FMKTokenSupervisor>,void>::run()const (d3320, d3620, fe189b18, fcd4ba00, d3538, 0) + 20 fd645a54 void RWThreadFunctionImp::run() (d3520, 0, fb4fbe24, 1, 1, ff267070) + 6c fd62a688 void RWRunnableImp::exec() (d3520, fb4fbec8, fd6462f4, 0, 0, fb4fbec4) + d0 fd6462a0 void RWThreadImp::exec() (d3520, 0, 0, 0, fcd4ba00, 1) + 30 fd646238 RWThreadImp_entry (d3520, fb4fc000, 0, 0, fd646234, 1) + 4 fd2c49fc _lwp_start (0, 0, 0, 0, 0, 0) The pflags show that the #19 receives SIGSEGV signal: /18: flags = STOPPED lwp_park(0x0,0x0,0x0) why = PR_SUSPENDED /19: flags = 0 sigmask = 0xffffbefc,0x0000ffff cursig = SIGSEGV /20: flags = STOPPED lwp_park(0x0,0x0,0x0) why = PR_SUSPENDED

wodaxia wrote: > One core was generated during execution. According to the pstack > result, it seems that the core came from thread 19 which was running > operator() method. [...] > ff319df4 bool > std::less<FMK_String>::operator() > ( const FMK_String&, const FMK_String&) const > (ff27fce9, 4d68, fbbf9be4, 46, cd5d4, ffc9ff46) + 14 [slightly reformatted] This is the operator() of std::less, a class that is used as comparator in a tree structure. > I was confused because most core stack I have seens was caused by > kill, abort or others, The last method called in thread was operator() > instead of kill, abort... [...] > The pflags show that the #19 receives SIGSEGV signal: Well, if a thread receives a signal, it could be anywhere, not just in kill/abort. Probably that is caused by a broken implementation of std::less<FMK_String> (does it treat null pointers/strings correctly?) or invalid input parameters to it. My guess is that something fishy is going on with the FMK_String type (why does that use a prefix instead of a namespace, BTW?) in combination with threads. This caused an invalid string to be stored in the tree where std::less is used to sort elements. Good luck! Uli

Thanks for your reply. Actually, the map in my code is declared as FMK_TValMap< FMK_String, TokenElement, std::less<FMK_String> > FMK_TValMap and FMK_String directly inherit RWClass RWTValMap and RWCString respectively. My code segment is: const FMK_String tokenId(userContext.SSOTokenIDStr.val); ActionListener managerAction(this, &TokenManager<S>::OnTokenNotify); Supervisor* supvr =3D NULL; Attach(userContext, 1, managerAction, supvr); trace.write("Token listener successfully setup for " << tokenId); // This has been print out.. FMK_Mutex::LockGuard guard(mutex_); if (tokenMap_.contains(tokenId)) { } On May 7, 12:57=A0pm, Ulrich Eckhardt <dooms...@knuut.de> wrote: > wodaxia wrote: > > Onecorewas generated during execution. According to the pstack > > result, it seems that thecorecame from thread 19 which was running > > operator() method. > [...] > > =A0ff319df4 bool > > =A0std::less<FMK_String>::operator() > > =A0 =A0 =A0( const FMK_String&, const FMK_String&) const > > =A0(ff27fce9, 4d68, fbbf9be4, 46, cd5d4, ffc9ff46) + 14 > > [slightly reformatted] > > This is the operator() of std::less, a class that is used as comparator i= n a > tree structure. > > > > > I was confused because mostcorestack I have seens was caused by > > kill, abort or others, The last method called in thread was operator() > > instead of kill, abort... > [...] > > The pflags show that the #19 receives SIGSEGV signal: > > Well, if a thread receives a signal, it could be anywhere, not just in > kill/abort. Probably that is caused by a broken implementation of > std::less<FMK_String> (does it treat null pointers/strings correctly?) or > invalid input parameters to it. [Shijie]: Do you mean that the parameter FMK_String is NULL(The FMK_String here is const object) or the map is NULL? Before any operation on map, one exclusive lock has been set on map, Also, no operation on map was done by other thread at that time. > My guess is that something fishy is going on with the FMK_String type (wh= y > does that use a prefix instead of a namespace, BTW?) in combination with > threads. This caused an invalid string to be stored in the tree where > std::less is used to sort elements. [Shijie]: FMK_String is user defined class. Do you mean that the FMK_string stored in the map may be invalid? I don;t think the input parameter for map.contains be NULL. > > Good luck! > > Uli

On May 5, 8:06=A0am, wodaxia <xushijie1...@yahoo.com.cn> wrote: > Hi, all, > One core was generated during execution. According to the pstack > result, it seems that the core came from thread 19 which was running > operator() method. > > I was confused because most core stack I have seens was caused by > kill, abort or others, The last method called in thread was operator() > instead of kill, abort... > > This core was generated once, and I was unable to reproduce it again. > Could anyone give me some hints on how to finger out the root cause? Check to see what the two arguments to std::less<FMK_String>::operator () look like -- are they valid? Also, it looks to me as though your specialization of std::map is wrong: std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::les= s<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenElement= > > It should be: std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::les= s<FMK_String>,std::allocator<std::pair<const FMK_String, TokenManager<FMKTokenSupervisor>::TokenElement> > > I.e., the Allocator argument should be std::allocator<std::pair<const T, U> > where T is FMK_String and U is TokenManager<FMKTokenSupervisor>::TokenElement. See: http://docs.sun.com/source/819-3704/map_8018.htm The Sun C++ Standard Library is old and has a number of quirks and limitations. You might want to check the Sun Studio documentation or ask on the Sun Studio C++ Forum: http://forums.sun.com/forum.jspa?forumID=3D850&start=3D0 The forum is also an excellent place to ask for help with these kinds of problems.

Thanks for your comments. I will go and check third parameter std::allocator. Regarding to this pro, I have found that the static instance of this class, one member of which is this trouble tokenMap_ could be created concurrently. I fixed this problem. But I was unable to verify my fix because of difficulty in reproducing this issue. Thanks Shijie Xu On May 18, 7:09=A0am, Martin Sebor <mse...@gmail.com> wrote: > On May 5, 8:06=A0am, wodaxia <xushijie1...@yahoo.com.cn> wrote: > > > Hi, all, > > One core was generated during execution. According to the pstack > > result, it seems that the core came from thread 19 which was running > > operator() method. > > > I was confused because most core stack I have seens was caused by > > kill, abort or others, The last method called in thread was operator() > > instead of kill, abort... > > > This core was generated once, and I was unable to reproduce it again. > > Could anyone give me some hints on how to finger out the root cause? > > Check to see what the two arguments to std::less<FMK_String>::operator > () > look like -- are they valid? > > Also, it looks to me as though your specialization of std::map is > wrong: > > std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::l= ess<FMK_String>,std::allocator<TokenManager<FMKTokenSupervisor>::TokenEleme= nt> > > > > It should be: > > std::map<FMK_String,TokenManager<FMKTokenSupervisor>::TokenElement,std::l= ess<FMK_String>,std::allocator<std::pair<const > FMK_String, TokenManager<FMKTokenSupervisor>::TokenElement> > > > > I.e., the Allocator argument should be std::allocator<std::pair<const > T, U> > where T is FMK_String and U is > TokenManager<FMKTokenSupervisor>::TokenElement. > See:http://docs.sun.com/source/819-3704/map_8018.htm > > The Sun C++ Standard Library is old and has a number of quirks and > limitations. You might want to check the Sun Studio documentation or > ask on the Sun Studio C++ Forum:http://forums.sun.com/forum.jspa?forumID= =3D850&start=3D0 > > The forum is also an excellent place to ask for help with these kinds > of problems.

Similiar Articles:

analyze core files - comp.unix.solaris
I have a core file generated by ns-slapd which I want to analyse and when I give ... the memory was corrupted elsewhere, so the call stack of > the core file does not help.

1653 - At least one timing constraint is impossible to meet - comp ...
The clock I used is generated using the > clock generator IP core of xilinx. This IP core can support up to 15 clock. > I use its 4th out put .

Anyone used the Xilinx' floating point core? - comp.arch.fpga ...
Thanks! Auto generated ... Please help, Xilinx FIFO problem! - comp.arch.fpga... FIFOs included in the FSL IP core (inside the EDK HW IP ...

Generating core using .mif file - comp.arch.fpga
Generating core using .mif file - comp.arch.fpga... use .MIF to generated a rom using xilinx core ... A MIF data file is used to support HDL functional simulation of modules ...

Solaris Core --Signal 9 --not able to find anything useful - comp ...
Core was generated by `./mybin'. Program terminated with signal 9, Killed. #0 0x72ef4 in my ... Which wont help if you use a stripped program. >I think i faced signal 9 ...

Xilinx CoreGen fifo - ngdbuild error - comp.arch.fpga
I have a fifo IP generated the Xilinx CORE Generator. In my design, I make an instance of the ... patchadd fails when trying to patch patchadd.... please help ..... often ...

HP-UX 11.11 overwriting core file(s) - How to prevent? - comp.sys ...
I reported a core file from our product but it turns ... found after the next test are known to have been generated ... Of course, this won't help when the product or test ...

Xilinx System Generator - Multiple system generator block - comp ...
Generating core using .mif file - comp.arch.fpga... use .MIF to generated a rom using xilinx core generator. ... How to synthesyze a RAM block?? Help Me ...

Memory mapped segments in a core dump - comp.unix.solaris ...
I'm debugging a core dump on Solaris 9 with dbx. ... -- -Gary Mills- -Unix Support- -U of M Academic ... fpga Can anybody tell me how can I use .MIF to generated a ...

Incorporating CoreGen files in EDK 8.1 peripheral - comp.arch.fpga ...
The async_fifo.edn is generated on the fly through some HDL ... to point it out anyway: Make sure that the core ... Maybe synthesis on the fly can help, since Xilinx ...

Brains And Bots Deep Inside Yahoo's CORE Grab A Billion Clicks ...
... including at least one you usually can’t help but ... of rolling out the algorithm--it's nicknamed CORE (for ... among Yahoo partners' content, or the stories generated in ...

LAUSD Invests $24 Million in Assessment Platform -- THE Journal
... assessment system designed to help boost student achievement through teacher-generated formative tests. The district signed a five-year, $24 Million contract with Core K12 ...

7/30/2012 1:07:57 PM