| cfkeem 2006-05-09, 1:16 am |
| Brief description of system environment:
-----------------------------------------
1. Environment: Websphere 5.1, Struts, EJB 2.0, Sun Solaris
2. Database: SQL Server 2000, Windows server 2003
3. Using EJB's CMP and only 1 report using JDBC Call
4. Live condition:
a) Users do data entry : 10 ppls
b) Users using barcode scanners to capture data (via EJB): 3ppls (each transaction around 100 data scans)
c) Users using barcode scanners to capture data (via web-services - another application)
Problems:
----------
1. System slows down for every half/ an hour.
I discovered there was deadlocks happened on both application logs/ database logs.
Below are message from websphere:
----------------------------------
[5/6/06 23:06:52:626 SGT] 1b3f1a SystemErr R at com.ibm.CORBA.iiop.UtilDelegateImpl.mapSystemException(UtilDelegateImpl.java:162)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at com.ibm.CORBA.iiop.UtilDelegateImpl.wrapException(UtilDelegateImpl.java:684)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at javax.rmi.CORBA.Util.wrapException(Util.java:277)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.ejb.facade._CutPieceRequestLineFacade_Stub. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(_CutPieceRequestLineFa
cade_Stub.java:330)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.delegate.ejb.CutPieceRequestLineEJBDelegateImpl. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(CutPieceRequestLineEJB
DelegateImpl.java:150)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.web.actions. CutPieceRequestLineBuyerPOColorSizeListA
ction. execute(CutPieceRequestLineBuyerPOColorS
izeListAction.java:68)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at org.apache.struts.action.RequestProcessor.processActionPerform(RequestProcessor.java:484)
[5/6/06 23:06:52:627 SGT] 1b3f1a SystemErr R at org.apache.struts.action.RequestProcessor.process(RequestProcessor.java:274)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at org.apache.struts.action.ActionServlet.process(ActionServlet.java:1482)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at org.apache.struts.action.ActionServlet.doGet(ActionServlet.java:507)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.StrictServletInstance.doService(StrictServletInstance.java:110)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet._service(StrictLifecycleServlet.java:174)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.IdleServletState.service(StrictLifecycleServlet.java:313)
[5/6/06 23:06:52:628 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.StrictLifecycleServlet.service(StrictLifecycleServlet.java:116)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.ServletInstance.service(ServletInstance.java:283)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.ValidServletReferenceState.dispatch(ValidServletReferenceState.java:42)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.servlet.ServletInstanceReference.dispatch(ServletInstanceReference.java:40)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:76)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.filter.URLAccessFilter.doFilter(URLAccessFilter.java:200)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.filter.FilterInstanceWrapper.doFilter(FilterInstanceWrapper.java:132)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.filter.WebAppFilterChain.doFilter(WebAppFilterChain.java:71)
[5/6/06 23:06:52:629 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher. handleWebAppDispatch(WebAppRequestDispat
cher.java:1010)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.dispatch(WebAppRequestDispatcher.java:592)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.webapp.WebAppRequestDispatcher.forward(WebAppRequestDispatcher.java:204)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.srt.WebAppInvoker.doForward(WebAppInvoker.java:125)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.srt.WebAppInvoker.handleInvocationHook(WebAppInvoker.java:286)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.cache.invocation.CachedInvocation.handleInvocation(CachedInvocation.java:71)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.srp.ServletRequestProcessor.dispatchByURI(ServletRequestProcessor.java:182)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.oselistener.OSEListenerDispatcher.service(OSEListener.java:334)
[5/6/06 23:06:52:630 SGT] 1b3f1a SystemErr R at com.ibm.ws.webcontainer.http.HttpConnection.handleRequest(HttpConnection.java:56)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ws.http.HttpConnection.readAndHandleRequest(HttpConnection.java:615)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ws.http.HttpConnection.run(HttpConnection.java:439)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:912)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R Caused by: java.rmi.RemoteException: ; nested exception is:
java.sql.SQLException: [IBM][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 64) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun the transaction.
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.RemoteExceptionMappingStrategy. mapEJBException(RemoteExceptionMappingSt
rategy.java:379)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.RemoteExceptionMappingStrategy. mapException(RemoteExceptionMappingStrat
egy.java:112)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.RemoteExceptionMappingStrategy. setUncheckedException(RemoteExceptionMap
pingStrategy.java:202)
[5/6/06 23:06:52:631 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.EJSDeployedSupport. setUncheckedException(EJSDeployedSupport
.java:296)
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.ejb.facade. EJSRemoteStatelessCutPieceRequestLineFac
ade_8cceab46. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(EJSRemoteStatelessCutP
ieceRequestLineFacad
e_8cceab46.java:125)
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.ejb.facade._CutPieceRequestLineFacade_Stub. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(_CutPieceRequestLineFa
cade_Stub.java:323)
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R ... 31 more
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R Caused by: java.sql.SQLException: [IBM][SQLServer JDBC Driver][SQLServer]Transaction (Process ID 64) was deadlocked on lock resources with another process and has been chosen as the deadlock victim. Rerun
the transaction.
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.base.BaseExceptions.createException(Unknown Source)
[5/6/06 23:06:52:632 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.base.BaseExceptions.getException(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRequest.processErrorToken(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRequest.processReplyToken(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRPCRequest.processReplyToken(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRPCCursorExecuteRequest.processReplyToken(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRequest.processReply(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRequest.getRow(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRPCCursorExecuteRequest.getRow(Unknown Source)
[5/6/06 23:06:52:633 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.tds.TDSRPCCursorExecuteRequest.fetchNext(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.sqlserver.SQLServerImplResultSetServerSideCursor.next(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbc.base.BaseResultSet.next(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.websphere.jdbcx.base.BaseResultSetWrapper.next(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.ws.rsadapter.cci.WSRdbResultSetImpl.next(WSRdbResultSetImpl.java:3004)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.ws.ejbpersistence.dataaccess.RawBeanData.next(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.ws.ejbpersistence.dataaccess.ResultCollectionImpl.processRow(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.ws.ejbpersistence.dataaccess.AbstractResultCollection.get(Unknown Source)
[5/6/06 23:06:52:634 SGT] 1b3f1a SystemErr R at com.ibm.ws.ejbpersistence.dataaccess.ResultCollectionIteratorImpl.next(Unknown Source)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultServerImpl.nextPrimaryKey(FinderResultServerImpl.java:930)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultServerImpl.getWrapperBase(FinderResultServerImpl.java:542)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultServerImpl.getNextWrapper(FinderResultServerImpl.java:681)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultClientBase_Local. readAheadNextElement(FinderResultClientB
ase_Local.java:161)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultClientBase_Local.<init>(FinderResultClientBase_Local.java:86)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultClientIterator_Local.<init>(FinderResultClientIterator_Local.java:46)
[5/6/06 23:06:52:635 SGT] 1b3f1a SystemErr R at com.ibm.ejs.container.finder.FinderResultClientCollection_Local. iterator(FinderResultClientCollection_Lo
cal.java:128)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.appservice.CutPieceRequestLineAppServiceImpl. getTotalOtherRequestedQuantity(CutPieceR
equestLineAppServiceImpl.java:1808)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.appservice.CutPieceRequestLineAppServiceImpl. getRequestStatus(CutPieceRequestLineAppS
erviceImpl.java:1738)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.appservice.CutPieceRequestLineAppServiceImpl. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(CutPieceRequestLineApp
ServiceImpl.java:870)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.ejb.facade.CutPieceRequestLineFacadeBean. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(CutPieceRequestLineFac
adeBean.java:51)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R at com.oceanskyintl.optimizer.business.ejb.facade. EJSRemoteStatelessCutPieceRequestLineFac
ade_8cceab46. getAllCutPieceRequestLineByCuttingSchedu
leCutPieceRequest(EJSRemoteStatelessCutP
ieceRequestLineFacad
e_8cceab46.java:115)
[5/6/06 23:06:52:636 SGT] 1b3f1a SystemErr R ... 32 more
Thank you.
Keem
|