Results 1 to 8 of 8
  1. #1
    Join Date
    Mar 2008
    Posts
    136

    Unanswered: no resources to create process or thread

    Hi All,
    Please help to resolve this.

    Linux Machine,5 GB RAM.
    db2level: B21085I Instance "dt2dtdev" uses "64" bits and DB2 code release "SQL09053"
    with level identifier "06040107".
    Informational tokens are "DB2 v9.5.0.3", "special_21393", "MI00269_21393", and
    Fix Pack "3a".
    Product is installed at "/opt/ibm/db2/V9.5".
    In notification file:
    2012-02-14-20.56.05.136686 Instance:dt2dtdev Node:000
    PID:18575(db2tcpcm) TID:2944395616 Appid:none
    base sys utilities sqeAgentServices::RequestAgent Probe:100

    ADM7519W DB2 could not allocate an agent. The SQLCODE is "-1225".
    2012-02-14-20.56.05.136928 Instance:dt2dtdev Node:000
    PID:18575(db2tcpcm) TID:2944395616 Appid:none
    common communication sqlcctcpconnmgr_child Probe:125

    ADM7009E An error was encountered in the "TCPIP" protocol support. A possible
    cause is that the maximum number of agents has been exceeded.
    
    in db2diag file:


    2012-02-14-04.08.30.512853-480 E1378E408 LEVEL: Error (OS)
    PID : 9404 TID : 183291079008 PROC : db2sysc
    INSTANCE: dt2test NODE : 000
    EDUID : 56 EDUNAME: db2loggw (DEVO2)
    FUNCTION: DB2 UDB, oper system services, sqloNLCKLock, probe:150
    MESSAGE : ZRC=0x8300001C=-2097151972
    CALLED : OS, -, semget OSERR: ENOSPC (28)

    2012-02-14-04.08.30.514452-480 I1787E411 LEVEL: Error
    PID : 9404 TID : 183291079008 PROC : db2sysc
    INSTANCE: dt2test NODE : 000
    EDUID : 56 EDUNAME: db2loggw (DEVO2)
    FUNCTION: DB2 UDB, database utilities, sqluhAllocateHandleVerifyHistoryFilesAnd, probe:1103
    MESSAGE : ZRC=0x8300001C=-2097151972
    DATA #1 : Sqlcode, PD_TYPE_SQLCODE, 4 bytes
    -2510

    2012-02-14-04.08.30.514598-480 I2199E359 LEVEL: Error
    PID : 9404 TID : 183291079008 PROC : db2sysc
    INSTANCE: dt2test NODE : 000
    EDUID : 56 EDUNAME: db2loggw (DEVO2)
    FUNCTION: DB2 UDB, database utilities, sqluhOpen, probe:1496
    MESSAGE : SQL2510N An operating system semaphore error occurred.

    2012-02-14-04.08.33.203740-480 I8166E389 LEVEL: Severe
    PID : 9404 TID : 183333022048 PROC : db2sysc
    INSTANCE: dt2test NODE : 000
    EDUID : 15 EDUNAME: db2tcpcm
    FUNCTION: DB2 UDB, global services, sqzEDUObj:tartEDU, probe:10
    RETCODE : ZRC=0x870F00F2=-2029059854=SQLO_NORES
    "no resources to create process or thread"

    db2 get dbm cfg
    Database Manager Configuration

    Node type = Database Server with local and remote clients

    Database manager configuration release level = 0x0c00

    CPU speed (millisec/instruction) (CPUSPEED) = 3.306410e-07

    Max number of concurrently active databases (NUMDB) = 8
    Federated Database System Support (FEDERATED) = YES
    Transaction processor monitor name (TP_MON_NAME) =

    Default charge-back account (DFT_ACCOUNT_STR) =

    Java Development Kit installation path (JDK_PATH) = /home/dt2test/sqllib/java/jdk64

    Diagnostic error capture level (DIAGLEVEL) = 3
    Notify Level (NOTIFYLEVEL) = 3
    Diagnostic data directory path (DIAGPATH) = /home/dt2test/sqllib/db2dump

    Default database monitor switches
    Buffer pool (DFT_MON_BUFPOOL) = ON
    Lock (DFT_MON_LOCK) = ON
    Sort (DFT_MON_SORT) = ON
    Statement (DFT_MON_STMT) = ON
    Table (DFT_MON_TABLE) = OFF
    Timestamp (DFT_MON_TIMESTAMP) = ON
    Unit of work (DFT_MON_UOW) = ON
    Monitor health of instance and databases (HEALTH_MON) = ON

    SYSADM group name (SYSADM_GROUP) = DB2IADM1
    SYSCTRL group name (SYSCTRL_GROUP) =
    SYSMAINT group name (SYSMAINT_GROUP) =
    SYSMON group name (SYSMON_GROUP) =

    Client Userid-Password Plugin (CLNT_PW_PLUGIN) =
    Client Kerberos Plugin (CLNT_KRB_PLUGIN) =
    Group Plugin (GROUP_PLUGIN) =
    GSS Plugin for Local Authorization (LOCAL_GSSPLUGIN) =
    Server Plugin Mode (SRV_PLUGIN_MODE) = UNFENCED
    Server List of GSS Plugins (SRVCON_GSSPLUGIN_LIST) =
    Server Userid-Password Plugin (SRVCON_PW_PLUGIN) =
    Server Connection Authentication (SRVCON_AUTH) = NOT_SPECIFIED
    Cluster manager (CLUSTER_MGR) =

    Database manager authentication (AUTHENTICATION) = SERVER
    Cataloging allowed without authority (CATALOG_NOAUTH) = NO
    Trust all clients (TRUST_ALLCLNTS) = YES
    Trusted client authentication (TRUST_CLNTAUTH) = CLIENT
    Bypass federated authentication (FED_NOAUTH) = NO

    Default database path (DFTDBPATH) = /home/dt2test

    Database monitor heap size (4KB) (MON_HEAP_SZ) = AUTOMATIC
    Java Virtual Machine heap size (4KB) (JAVA_HEAP_SZ) = 4096
    Audit buffer size (4KB) (AUDIT_BUF_SZ) = 0
    Size of instance shared memory (4KB) (INSTANCE_MEMORY) = AUTOMATIC
    Backup buffer default size (4KB) (BACKBUFSZ) = 2048
    Restore buffer default size (4KB) (RESTBUFSZ) = 2048

    Agent stack size (AGENT_STACK_SZ) = 1024
    Sort heap threshold (4KB) (SHEAPTHRES) = 51200

    Directory cache support (DIR_CACHE) = YES

    Application support layer heap size (4KB) (ASLHEAPSZ) = 15
    Max requester I/O block size (bytes) (RQRIOBLK) = 32767
    Query heap size (4KB) (QUERY_HEAP_SZ) = 1000

    Workload impact by throttled utilities(UTIL_IMPACT_LIM) = 10

    Priority of agents (AGENTPRI) = SYSTEM
    Agent pool size (NUM_POOLAGENTS) = AUTOMATIC
    Initial number of agents in pool (NUM_INITAGENTS) = 5
    Max number of coordinating agents (MAX_COORDAGENTS) = AUTOMATIC
    Max number of client connections (MAX_CONNECTIONS) = AUTOMATIC

    Keep fenced process (KEEPFENCED) = NO
    Number of pooled fenced processes (FENCED_POOL) = AUTOMATIC
    Initial number of fenced processes (NUM_INITFENCED) = 0

    Index re-creation time and redo index build (INDEXREC) = RESTART

    Transaction manager database name (TM_DATABASE) = 1ST_CONN
    Transaction resync interval (sec) (RESYNC_INTERVAL) = 180

    SPM name (SPM_NAME) =
    SPM log size (SPM_LOG_FILE_SZ) = 256
    SPM resync agent limit (SPM_MAX_RESYNC) = 20
    SPM log path (SPM_LOG_PATH) =

    TCP/IP Service name (SVCENAME) = DB2_dt2dtdev
    Discovery mode (DISCOVER) = SEARCH
    Discover server instance (DISCOVER_INST) = ENABLE

    Maximum query degree of parallelism (MAX_QUERYDEGREE) = ANY
    Enable intra-partition parallelism (INTRA_PARALLEL) = YES

    No. of int. communication buffers(4KB)(FCM_NUM_BUFFERS) = AUTOMATIC
    No. of int. communication channels (FCM_NUM_CHANNELS) = AUTOMATIC
    db2start/db2stop timeout (min) (START_STOP_TIME) = 10


    db2 get snapshot for database manager

    Database Manager Snapshot

    Node type = Database Server with local and remote clients
    Instance name = dt2test
    Number of database partitions in DB2 instance = 1
    Database manager status = Active

    Product name = DB2 v9.5.0.3
    Service level = special_21393 (MI00269_21393)

    Private Sort heap allocated = 0
    Private Sort heap high water mark = 10240
    Post threshold sorts = 0
    Piped sorts requested = 177
    Piped sorts accepted = 177

    Start Database Manager timestamp = 02/14/2012 20:46:04.631295
    Last reset timestamp =
    Snapshot timestamp = 02/14/2012 22:10:45.429377

    Remote connections to db manager = 4
    Remote connections executing in db manager = 0
    Local connections = 2
    Local connections executing in db manager = 0
    Active local databases = 1

    High water mark for agents registered = 11
    Agents registered = 8
    Idle agents = 0

    Committed private Memory (Bytes) = 11665408

    Switch list for db partition number 0
    Buffer Pool Activity Information (BUFFERPOOL) = ON 02/14/2012 20:46:04.631295
    Lock Information (LOCK) = ON 02/14/2012 20:46:04.631295
    Sorting Information (SORT) = ON 02/14/2012 20:46:04.631295
    SQL Statement Information (STATEMENT) = ON 02/14/2012 20:46:04.631295
    Table Activity Information (TABLE) = OFF
    Take Timestamp Information (TIMESTAMP) = ON 02/14/2012 20:46:04.631295
    Unit of Work Information (UOW) = ON 02/14/2012 20:46:04.631295

    Agents assigned from pool = 39
    Agents created from empty pool = 21
    Agents stolen from another application = 0
    High water mark for coordinating agents = 11
    Hash joins after heap threshold exceeded = 0
    OLAP functions after heap threshold exceeded = 0

    Total number of gateway connections = 0
    Current number of gateway connections = 0
    Gateway connections waiting for host reply = 0
    Gateway connections waiting for client request = 0
    Gateway connection pool agents stolen = 0

    Node FCM information corresponds to = 0
    Free FCM buffers = 895
    Free FCM buffers low water mark = 895
    Free FCM channels = 895
    Free FCM channels low water mark = 893


    Memory usage for database manager:

    Node number = 0
    Memory Pool Type = Other Memory
    Current size (bytes) = 26804224
    High water mark (bytes) = 26869760
    Configured size (bytes) = 52756480

    Node number = 0
    Memory Pool Type = FCMBP Heap
    Current size (bytes) = 12189696
    High water mark (bytes) = 12189696
    Configured size (bytes) = 12517376

    Node number = 0
    Memory Pool Type = Database Monitor Heap
    Current size (bytes) = 327680
    High water mark (bytes) = 327680
    Configured size (bytes) = 1048576
    ipcs -l

    ------ Shared Memory Limits --------
    max number of segments = 4096
    max seg size (kbytes) = 5242880
    max total shared memory (kbytes) = 8388608
    min seg size (bytes) = 1

    ------ Semaphore Limits --------
    max number of arrays = 1024
    max semaphores per array = 250
    max semaphores system wide = 256000
    max ops per semop call = 32
    semaphore max value = 32767

    ------ Messages: Limits --------
    max queues system wide = 1024
    max size of message (bytes) = 65536
    default max size of queue (bytes) = 65536
    Regards
    Pawan Kumar

  2. #2
    Join Date
    Apr 2006
    Location
    Belgium
    Posts
    2,514
    Provided Answers: 11
    have you checked how many connections have already been taken
    maybe you hit the limit of tcpip definition where you can not create any new conn anymore because of durty/none cleanup and you always request new connections
    Best Regards, Guy Przytula
    Database Software Consultant
    Good DBAs are not formed in a week or a month. They are created little by little, day by day. Protracted and patient effort is needed to develop good DBAs.
    Spoon feeding : To treat (another) in a way that discourages independent thought or action, as by overindulgence.
    DB2 UDB LUW Certified V7-V8-V9-V9.7-V10.1-V10.5 DB Admin - Advanced DBA -Dprop..
    Information Server Datastage Certified
    http://www.infocura.be

  3. #3
    Join Date
    Mar 2008
    Posts
    136
    how I can get the limit of tcpip definition???

    This machine is being used by only one person(QA).

    I stop all the application server and db2 also then restart db2 and restart the all application server. Whenever application server try to make connection it fails.

    so its not possible that many users are trying to connect. even we are not doing load testing.
    Regards
    Pawan Kumar

  4. #4
    Join Date
    Apr 2006
    Location
    Belgium
    Posts
    2,514
    Provided Answers: 11
    the problem is with db2loggw : log writer
    check the logpath in db cfg
    check you own this path and can write to this path
    enough space is available
    is db activated ?
    have you tried manual connect : local - remote ?
    publishing error data is not enough - some pd/psi from your part is also needed
    Best Regards, Guy Przytula
    Database Software Consultant
    Good DBAs are not formed in a week or a month. They are created little by little, day by day. Protracted and patient effort is needed to develop good DBAs.
    Spoon feeding : To treat (another) in a way that discourages independent thought or action, as by overindulgence.
    DB2 UDB LUW Certified V7-V8-V9-V9.7-V10.1-V10.5 DB Admin - Advanced DBA -Dprop..
    Information Server Datastage Certified
    http://www.infocura.be

  5. #5
    Join Date
    Mar 2008
    Posts
    136
    Space is available.
    earlier I was able connect but its failing now....
    db2 connect to DEVO2 user test using test
    DB21017E The Command Line Processor encountered a system error with the
    front-end process output queue. Reason code = -2029060029.

    2012-02-15-04.47.15.450162-480 I5201E538 LEVEL: Error
    PID : 16618 TID : 183307856224 PROC : db2sysc
    INSTANCE: db2test NODE : 000 DB : DEVO2
    APPHDL : 0-43885 APPID: 10.11.102.30.45760.120215194400
    AUTHID : test
    EDUID : 179 EDUNAME: db2agent (DEVO2)
    FUNCTION: DB2 UDB, buffer pool services, sqlbReadBufferPoolTable, probe:900
    MESSAGE : ZRC=0x850F0006=-2062614522=SQLO_FHNL "TOO MANY OPEN FILES"
    DIA8306C Too many files were opened.

    2012-02-15-04.47.15.451179-480 E5740E1009 LEVEL: Error (OS)
    PID : 16618 TID : 183307856224 PROC : db2sysc
    INSTANCE: dt2test NODE : 000 DB : DEVO2
    APPHDL : 0-43885 APPID: 10.11.102.30.45760.120215194400
    AUTHID : test
    EDUID : 179 EDUNAME: db2agent (DEVO2)
    FUNCTION: DB2 UDB, oper system services, sqloopenp, probe:80
    MESSAGE : ZRC=0x850F0006=-2062614522=SQLO_FHNL "TOO MANY OPEN FILES"
    DIA8306C Too many files were opened.
    CALLED : OS, -, open OSERR: EMFILE (24)
    DATA #1 : Codepath, 8 bytes
    4:14:15:23:37
    DATA #2 : File name, 53 bytes
    /home/db2test/db2test/NODE0000/SQL00003///SQLDBCONF
    DATA #3 : Bitmask, 4 bytes
    0x0000004C
    DATA #4 : Hex integer, 4 bytes
    0x000001A0
    DATA #5 : signed integer, 4 bytes
    0
    DATA #6 : signed integer, 4 bytes
    0
    DATA #7 : String, 105 bytes
    Search for ossError*Analysis probe point after this log entry for further
    self-diagnosis of this problem.

    2012-02-15-04.47.15.451542-480 I6750E2264 LEVEL: Error (OS)
    Regards
    Pawan Kumar

  6. #6
    Join Date
    Apr 2006
    Location
    Belgium
    Posts
    2,514
    Provided Answers: 11
    as indicated before : error publishing is not enough - we can NOT debug for you..
    we can just give some hints and info
    ulimit ??
    the message is very clear : TOO MANY OPEN FILES
    Best Regards, Guy Przytula
    Database Software Consultant
    Good DBAs are not formed in a week or a month. They are created little by little, day by day. Protracted and patient effort is needed to develop good DBAs.
    Spoon feeding : To treat (another) in a way that discourages independent thought or action, as by overindulgence.
    DB2 UDB LUW Certified V7-V8-V9-V9.7-V10.1-V10.5 DB Admin - Advanced DBA -Dprop..
    Information Server Datastage Certified
    http://www.infocura.be

  7. #7
    Join Date
    Aug 2008
    Location
    Toronto, Canada
    Posts
    2,367
    It's better if you attach a complete db2diag.log.

    The errors are coming back from the OS:

    -> CALLED : OS, -, semget OSERR: ENOSPC (28)
    -> CALLED : OS, -, open OSERR: EMFILE (24)


    ENOSPC from semget -> kernel parameters (specifically semaphores) need to be increased.

    See: IBM DB2 LUW: the db2diag.log is reporting an ENOSPC (28) error message - United States

    When you upgrade to v9.6 FP6+, they will be increased by db2 as needed.


    EMFILE from open -> check max file limit for the instance id (ulimit)

  8. #8
    Join Date
    Mar 2008
    Posts
    136
    Thanks All!
    After restarting the machine all issues went away.
    Regards
    Pawan Kumar

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •